[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:54:48 dist-test-slave-1x4g krb5kdc[1879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:54:48 dist-test-slave-1x4g krb5kdc[1879](info): set up 2 sockets
Aug 08 22:54:48 dist-test-slave-1x4g krb5kdc[1879](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 22:54:49.428360 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:54:49 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:54:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:54:49.566030 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37863
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:49.662624 1892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:49.662916 1892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:49.662962 1892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:49.664319 1892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:54:49.664395 1892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:49.664422 1892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:49.664443 1892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:54:49.664474 1892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:54:49.666416 1892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:49.666744 1892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:49.666997 1892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:49.672920 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:49.672966 1898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:49.673012 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:49.673362 1892 server_base.cc:1034] running on GCE node
I20260808 22:54:49.673696 1892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:49.674027 1892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:49.675216 1892 hybrid_clock.cc:648] HybridClock initialized: now 1786229689675204 us; error 47 us; skew 500 ppm
Aug 08 22:54:49 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:49.679064 1892 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:54:49.679733 1892 webserver.cc:460] Webserver started at http://127.1.212.62:46857/ using document root <none> and password file <none>
I20260808 22:54:49.679943 1892 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:49.679991 1892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:49.680101 1892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:49.681633 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f01f89410b0d4c2a94897a33675b115d"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "6154d238cda1264a33339808140a0088"
server_key_iv: "919ec8af225e69ba63bf9df5cabf65af"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.682054 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f01f89410b0d4c2a94897a33675b115d"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "6154d238cda1264a33339808140a0088"
server_key_iv: "919ec8af225e69ba63bf9df5cabf65af"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.683992 1892 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:54:49.684921 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:49.685156 1892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:49.685284 1892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f01f89410b0d4c2a94897a33675b115d"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "6154d238cda1264a33339808140a0088"
server_key_iv: "919ec8af225e69ba63bf9df5cabf65af"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.685406 1892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:49.695153 1892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:49.697544 1892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:49.697777 1892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:49.705667 1892 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37863
I20260808 22:54:49.705717 1962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37863 every 8 connection(s)
I20260808 22:54:49.706205 1892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:54:49.707371 1963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:49.710351 1963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d: Bootstrap starting.
I20260808 22:54:49.711069 1963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:49.711427 1963 log.cc:824] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:49.712453 1963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d: No bootstrap required, opened a new log
I20260808 22:54:49.714393 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 1892
I20260808 22:54:49.714323 1963 raft_consensus.cc:348] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } }
I20260808 22:54:49.714495 1963 raft_consensus.cc:374] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:49.714532 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:54:49.714520 1963 raft_consensus.cc:729] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01f89410b0d4c2a94897a33675b115d, State: Initialized, Role: FOLLOWER
I20260808 22:54:49.714756 1963 consensus_queue.cc:260] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } }
I20260808 22:54:49.714840 1963 raft_consensus.cc:388] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:54:49.714864 1963 raft_consensus.cc:482] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:54:49.714946 1963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:49.715337 1872 external_mini_cluster.cc:1428] Setting key 4b7ef812e78b0c601919b2223e202aa2
I20260808 22:54:49.715608 1963 raft_consensus.cc:504] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } }
I20260808 22:54:49.715729 1963 leader_election.cc:302] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f01f89410b0d4c2a94897a33675b115d; no voters:
I20260808 22:54:49.715958 1963 leader_election.cc:288] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:54:49.715984 1966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:49.716112 1966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [term 1 LEADER]: Becoming Leader. State: Replica: f01f89410b0d4c2a94897a33675b115d, State: Running, Role: LEADER
I20260808 22:54:49.716194 1966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } }
I20260808 22:54:49.716264 1963 sys_catalog.cc:564] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:54:49.716552 1966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f01f89410b0d4c2a94897a33675b115d. Latest consensus state: current_term: 1 leader_uuid: "f01f89410b0d4c2a94897a33675b115d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } } }
I20260808 22:54:49.716663 1966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:49.716566 1967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f01f89410b0d4c2a94897a33675b115d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01f89410b0d4c2a94897a33675b115d" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37863 } } }
I20260808 22:54:49.716740 1967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:49.718067 1978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:54:49.719149 1978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:54:49.722023 1978 catalog_manager.cc:1347] Generated new cluster ID: 5dbd967a80ce4ad0901bcc0b60f240fa
I20260808 22:54:49.722082 1978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:54:49.730077 1978 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:54:49.730839 1978 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 22:54:49 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:49.733382 1978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f01f89410b0d4c2a94897a33675b115d: Generated new TSK 0
I20260808 22:54:49.733580 1978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:54:49.737129 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:49.717183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.1.212.62:37863)
0808 22:54:49.717481 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:49.717492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:49.718208 (+ 716us) server_negotiation.cc:407] Connection header received
0808 22:54:49.721141 (+ 2933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:49.721153 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:49.721199 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:49.721336 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:49.723529 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.725588 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.727278 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.728001 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.728129 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:49.732817 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:49.732823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:49.732823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:49.732837 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:49.734824 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:49.735433 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:49.735437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:49.735437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:49.735539 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:49.735875 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:49.735876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:49.735876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:49.736133 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:49.736250 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:49.736610 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 22:54:49.736750 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:54:49.768488 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:49.859347 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:49.859623 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:49.859671 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:49.861021 1989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:49.861091 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:49.861140 1989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:54:49.862782 1989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:49.863121 1989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:49.863376 1989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:49.868886 1997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:49.868963 1996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:49.868892 1999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:49.869197 1989 server_base.cc:1034] running on GCE node
I20260808 22:54:49.869371 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:49.869676 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:49.870831 1989 hybrid_clock.cc:648] HybridClock initialized: now 1786229689870808 us; error 39 us; skew 500 ppm
Aug 08 22:54:49 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:49.873566 1989 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:54:49.874158 1989 webserver.cc:460] Webserver started at http://127.1.212.1:42639/ using document root <none> and password file <none>
I20260808 22:54:49.874420 1989 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:49.874469 1989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:49.874658 1989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:49.875831 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "dfcdac62309749e5bb27bfb6b49fa66a"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "bcd22c4cf9794447640147a74f180ff7"
server_key_iv: "14faaf75adb4fe7027eadf5ad8076f25"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.876382 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "dfcdac62309749e5bb27bfb6b49fa66a"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "bcd22c4cf9794447640147a74f180ff7"
server_key_iv: "14faaf75adb4fe7027eadf5ad8076f25"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.878998 1989 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 22:54:49.879895 2006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:49.880077 1989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:49.880195 1989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "dfcdac62309749e5bb27bfb6b49fa66a"
format_stamp: "Formatted at 2026-08-08 22:54:49 on dist-test-slave-1x4g"
server_key: "bcd22c4cf9794447640147a74f180ff7"
server_key_iv: "14faaf75adb4fe7027eadf5ad8076f25"
server_key_version: "encryptionkey@0"
I20260808 22:54:49.880290 1989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:49.898203 1989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:49.900245 1989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:49.900499 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:49.900803 1989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:49.901208 1989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:49.901268 1989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:49.901304 1989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:49.901346 1989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:49.910339 1989 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:40871
I20260808 22:54:49.910385 2125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:40871 every 8 connection(s)
I20260808 22:54:49.910770 1989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:54:49.915441 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 1989
I20260808 22:54:49.915544 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:54:49.915716 1872 external_mini_cluster.cc:1428] Setting key 96f80666d3536e6d4e2b6d8d653225dd
Aug 08 22:54:49 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:49.922047 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:49.911634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:33153 (local address 127.1.212.62:37863)
0808 22:54:49.911705 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:49.911708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:49.911951 (+ 243us) server_negotiation.cc:407] Connection header received
0808 22:54:49.912455 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:49.912456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:49.912466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:49.912524 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:49.913742 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.914287 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.915363 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.915702 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.915779 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:49.918800 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:49.918801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:49.918801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:49.918809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:49.920449 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:49.920897 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:49.920898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:49.920899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:49.920977 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:49.921243 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:49.921244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:49.921244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:49.921428 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:49.921507 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:49.921788 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 22:54:49.921905 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:54:49.922060 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:49.911618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37863 (local address 127.1.212.1:33153)
0808 22:54:49.911801 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:49.911846 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:49.912355 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:49.912588 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:49.912591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:49.913371 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:49.913591 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:49.913596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.914389 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.914390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:49.915257 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:49.915260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:49.915818 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:49.915819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:49.915992 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:49.916691 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:49.916703 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:49.918699 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:49.920539 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:49.920547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:49.920549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:49.920827 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:49.921048 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:49.921048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:49.921048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:49.921158 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:49.921506 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:49.921512 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:49.921646 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 22:54:49.921730 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:54:49.921872 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260808 22:54:49.922679 2126 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37863
I20260808 22:54:49.922811 2126 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:49.923043 2126 heartbeater.cc:511] Master 127.1.212.62:37863 requested a full tablet report, sending...
I20260808 22:54:49.923787 1925 ts_manager.cc:194] Registered new tserver with Master: dfcdac62309749e5bb27bfb6b49fa66a (127.1.212.1:40871)
I20260808 22:54:49.924494 1925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:33153
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:54:49.945535 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:50.035615 2131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:50.035858 2131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:50.035887 2131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:50.037226 2131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:50.037281 2131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:50.037312 2131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:54:50.038925 2131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:50.039239 2131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:50.039516 2131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:50.045243 2136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:50.045218 2140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:50.045364 2131 server_base.cc:1034] running on GCE node
W20260808 22:54:50.045369 2137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:50.045866 2131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:50.046288 2131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:50.047524 2131 hybrid_clock.cc:648] HybridClock initialized: now 1786229690047504 us; error 45 us; skew 500 ppm
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:50.050329 2131 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:54:50.050968 2131 webserver.cc:460] Webserver started at http://127.1.212.2:32777/ using document root <none> and password file <none>
I20260808 22:54:50.051232 2131 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:50.051304 2131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:50.051477 2131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:50.052558 2131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "839809d3038c40b6a468e3d4d2e6c088"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "3579e33d229bc1223bf367010c2589ba"
server_key_iv: "0014c5a057a7b506fe21362cf53a03e5"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.052956 2131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "839809d3038c40b6a468e3d4d2e6c088"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "3579e33d229bc1223bf367010c2589ba"
server_key_iv: "0014c5a057a7b506fe21362cf53a03e5"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.054528 2131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:50.055500 2148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.055845 2131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:50.055969 2131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "839809d3038c40b6a468e3d4d2e6c088"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "3579e33d229bc1223bf367010c2589ba"
server_key_iv: "0014c5a057a7b506fe21362cf53a03e5"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.056056 2131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:50.088481 2131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:50.091070 2131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:50.091274 2131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:50.091559 2131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:50.091965 2131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:50.092192 2131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.092262 2131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:50.092278 2131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.100108 2131 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:33759
I20260808 22:54:50.100152 2268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:33759 every 8 connection(s)
I20260808 22:54:50.100545 2131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:54:50.102285 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2131
I20260808 22:54:50.102414 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:54:50.102597 1872 external_mini_cluster.cc:1428] Setting key 1f53c91708b1eb0811d94d2b260fa390
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:50.111670 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.101419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:37677 (local address 127.1.212.62:37863)
0808 22:54:50.101495 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:50.101497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:50.101714 (+ 217us) server_negotiation.cc:407] Connection header received
0808 22:54:50.102353 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:50.102354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:50.102371 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:50.102419 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:50.103779 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.104302 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.105607 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.105930 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.106007 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.108396 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:50.108397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:50.108398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:50.108412 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:50.110018 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.110529 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.110531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.110532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.110602 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.110920 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.110921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.110921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.111060 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:50.111139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:50.111470 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 22:54:50.111572 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:54:50.111893 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.101393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37863 (local address 127.1.212.2:37677)
0808 22:54:50.101572 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:50.101620 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:50.102231 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:50.102531 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:50.102534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:50.103357 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:50.103674 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:50.103681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.104436 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.104437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:50.105469 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:50.105481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.106041 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.106042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:50.106219 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.106821 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:50.106831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:50.108274 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:50.110148 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:50.110159 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:50.110161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:50.110438 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:50.110695 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:50.110696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:50.110697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:50.110837 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:50.111141 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:50.111150 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:50.111315 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 22:54:50.111446 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 22:54:50.111637 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260808 22:54:50.112556 2269 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37863
I20260808 22:54:50.112689 2269 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:50.112932 2269 heartbeater.cc:511] Master 127.1.212.62:37863 requested a full tablet report, sending...
I20260808 22:54:50.113435 1925 ts_manager.cc:194] Registered new tserver with Master: 839809d3038c40b6a468e3d4d2e6c088 (127.1.212.2:33759)
I20260808 22:54:50.113839 1925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:37677
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:54:50.136246 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:50.228714 2275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:50.229023 2275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:50.229053 2275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:50.230234 2275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:50.230275 2275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:50.230304 2275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:54:50.231763 2275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:50.231983 2275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:50.232172 2275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:50.237665 2281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:50.237675 2282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:50.237903 2275 server_base.cc:1034] running on GCE node
W20260808 22:54:50.237957 2284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:50.238163 2275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:50.238431 2275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:50.239611 2275 hybrid_clock.cc:648] HybridClock initialized: now 1786229690239600 us; error 31 us; skew 500 ppm
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:50.242432 2275 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:54:50.242933 2275 webserver.cc:460] Webserver started at http://127.1.212.3:34293/ using document root <none> and password file <none>
I20260808 22:54:50.243187 2275 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:50.243259 2275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:50.243413 2275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:50.244577 2275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "13577aa0a0384867894925cd18446004"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "2dbd1bb1fe4a3e3658f0d95fe955be7f"
server_key_iv: "a9071fc6e0c54cbdef7726516f46f2da"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.245002 2275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "13577aa0a0384867894925cd18446004"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "2dbd1bb1fe4a3e3658f0d95fe955be7f"
server_key_iv: "a9071fc6e0c54cbdef7726516f46f2da"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.246551 2275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:50.247364 2292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.247543 2275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:50.247658 2275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "13577aa0a0384867894925cd18446004"
format_stamp: "Formatted at 2026-08-08 22:54:50 on dist-test-slave-1x4g"
server_key: "2dbd1bb1fe4a3e3658f0d95fe955be7f"
server_key_iv: "a9071fc6e0c54cbdef7726516f46f2da"
server_key_version: "encryptionkey@0"
I20260808 22:54:50.247736 2275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:50.267525 2275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:50.270061 2275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:50.270251 2275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:50.270539 2275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:50.270892 2275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:50.270949 2275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.270987 2275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:50.271034 2275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:50.278673 2275 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:34821
I20260808 22:54:50.278724 2410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:34821 every 8 connection(s)
I20260808 22:54:50.279124 2275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:54:50.283336 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2275
I20260808 22:54:50.283418 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:54:50.283700 1872 external_mini_cluster.cc:1428] Setting key 0797319bd460141c72daf375c37f9455
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:50.289875 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.279959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:41117 (local address 127.1.212.62:37863)
0808 22:54:50.280079 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:50.280081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:50.280311 (+ 230us) server_negotiation.cc:407] Connection header received
0808 22:54:50.280903 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:50.280905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:50.280918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:50.280980 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:50.282010 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.282539 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.283776 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.284150 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.284236 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.286807 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:50.286809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:50.286809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:50.286816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:50.288384 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.288865 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.288866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.288866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.288937 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.289234 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.289235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.289235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.289359 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:50.289416 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:50.289725 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:54:50.289807 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 22:54:50.290123 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.279908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37863 (local address 127.1.212.3:41117)
0808 22:54:50.280166 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:50.280217 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:50.280809 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:50.281040 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:50.281044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:50.281683 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:50.281901 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:50.281906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.282638 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.282639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:50.283681 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:50.283685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.284281 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.284283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:50.284465 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.285271 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:50.285281 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:50.286716 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:50.288480 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:50.288487 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:50.288489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:50.288731 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:50.289008 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:50.289008 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:50.289009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:50.289170 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:50.289439 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:50.289446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:50.289597 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 22:54:50.289689 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:54:50.289832 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":174,"threads_started":1}
I20260808 22:54:50.290695 2411 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37863
I20260808 22:54:50.290808 2411 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:50.291040 2411 heartbeater.cc:511] Master 127.1.212.62:37863 requested a full tablet report, sending...
I20260808 22:54:50.291527 1925 ts_manager.cc:194] Registered new tserver with Master: 13577aa0a0384867894925cd18446004 (127.1.212.3:34821)
I20260808 22:54:50.291934 1925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:41117
I20260808 22:54:50.296187 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:54:50.323364 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.314581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.1.212.1:40871)
0808 22:54:50.314773 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:50.314779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:50.314792 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:54:50.314824 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:50.314826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:50.314841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:50.314910 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:50.315490 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.316360 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.317276 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.317666 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.317778 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.320159 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:50.320171 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:50.320171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:50.320180 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:50.322020 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.322443 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.322448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.322448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.322531 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.322725 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.322726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.322726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.322910 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:50.323029 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:50.323101 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:54:50.323191 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
W20260808 22:54:50.323774 2036 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33594
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[1879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:54:50.350211 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:50.342533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.1.212.1:40871)
0808 22:54:50.342631 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:50.342633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:50.342646 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:54:50.342704 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:50.342705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:50.342720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:50.342781 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:50.343388 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.343895 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.344658 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:50.344985 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:50.345051 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:50.347417 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:50.347419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:50.347419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:50.347439 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:50.348970 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.349353 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.349355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.349355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.349421 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:50.349655 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:50.349656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:50.349656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:50.349773 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:50.349904 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:50.350005 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 22:54:50.350099 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:54:50.350940 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 1989
I20260808 22:54:50.359128 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2131
I20260808 22:54:50.366801 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2275
I20260808 22:54:50.373905 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 1892
2026-08-08T22:54:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2101 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[2427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[2427](info): set up 2 sockets
Aug 08 22:54:50 dist-test-slave-1x4g krb5kdc[2427](info): commencing operation
krb5kdc: starting...
W20260808 22:54:51.456169 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:54:51 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:54:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:54:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:54:51.592360 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37121
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:51.680145 2440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:51.680511 2440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:51.680559 2440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:51.681892 2440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:54:51.681944 2440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:51.681960 2440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:51.681973 2440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:54:51.682024 2440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:54:51.683676 2440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:51.683977 2440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:51.684249 2440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:51.689568 2445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:51.689571 2448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:51.689574 2446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:51.689839 2440 server_base.cc:1034] running on GCE node
I20260808 22:54:51.690099 2440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:51.690362 2440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:51.691545 2440 hybrid_clock.cc:648] HybridClock initialized: now 1786229691691523 us; error 31 us; skew 500 ppm
Aug 08 22:54:51 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:51.694356 2440 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:54:51.694846 2440 webserver.cc:460] Webserver started at http://127.1.212.62:42865/ using document root <none> and password file <none>
I20260808 22:54:51.695129 2440 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:51.695205 2440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:51.695358 2440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:51.696555 2440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "6807d12b801447af87f2adae254975d9"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "4cc7b751571df32e2ecc0e47cc61d119"
server_key_iv: "f5184abdcad1a59b19cad96329433f39"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.696991 2440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "6807d12b801447af87f2adae254975d9"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "4cc7b751571df32e2ecc0e47cc61d119"
server_key_iv: "f5184abdcad1a59b19cad96329433f39"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.698534 2440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:51.699369 2455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:51.699607 2440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:51.699716 2440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "6807d12b801447af87f2adae254975d9"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "4cc7b751571df32e2ecc0e47cc61d119"
server_key_iv: "f5184abdcad1a59b19cad96329433f39"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.699805 2440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:51.725579 2440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:51.728137 2440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:51.728323 2440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:51.732694 2440 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37121
I20260808 22:54:51.732739 2509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37121 every 8 connection(s)
I20260808 22:54:51.733198 2440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:54:51.734206 2510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:51.736925 2510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9: Bootstrap starting.
I20260808 22:54:51.737592 2510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:51.737918 2510 log.cc:824] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:51.738829 2510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9: No bootstrap required, opened a new log
I20260808 22:54:51.738979 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2440
I20260808 22:54:51.739086 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:54:51.739261 1872 external_mini_cluster.cc:1428] Setting key 66ed9d7b7d37d90404e6246de64bfb33
I20260808 22:54:51.741087 2510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } }
I20260808 22:54:51.741283 2510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:51.741326 2510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6807d12b801447af87f2adae254975d9, State: Initialized, Role: FOLLOWER
I20260808 22:54:51.741482 2510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } }
I20260808 22:54:51.741575 2510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:54:51.741621 2510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:54:51.741677 2510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:51.742893 2510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } }
I20260808 22:54:51.743001 2510 leader_election.cc:302] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6807d12b801447af87f2adae254975d9; no voters:
I20260808 22:54:51.743240 2510 leader_election.cc:288] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:54:51.743290 2515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:51.743458 2515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [term 1 LEADER]: Becoming Leader. State: Replica: 6807d12b801447af87f2adae254975d9, State: Running, Role: LEADER
I20260808 22:54:51.743532 2510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:54:51.743563 2515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } }
I20260808 22:54:51.743908 2515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6807d12b801447af87f2adae254975d9. Latest consensus state: current_term: 1 leader_uuid: "6807d12b801447af87f2adae254975d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } } }
I20260808 22:54:51.743995 2515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:51.743930 2517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6807d12b801447af87f2adae254975d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6807d12b801447af87f2adae254975d9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37121 } } }
I20260808 22:54:51.744146 2517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:51.744334 2521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:54:51.745185 2521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:54:51 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:51.748347 2521 catalog_manager.cc:1347] Generated new cluster ID: 13f54e84a58746149ec300ac03bab4a4
I20260808 22:54:51.748402 2521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:54:51.752008 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:51.739907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.1.212.62:37121)
0808 22:54:51.740114 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:51.740131 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:51.740158 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:54:51.741049 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:51.741057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:51.741111 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:51.741236 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:51.742097 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.743130 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.744298 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.744961 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.745082 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:51.747863 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:51.747868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:51.747869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:51.747884 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:51.749992 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:51.750378 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:51.750381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:51.750381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:51.750514 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:51.750688 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:51.750689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:51.750689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:51.751012 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:51.751154 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:51.751648 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 22:54:51.751787 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260808 22:54:51.781221 2521 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:54:51.782059 2521 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:54:51.788592 2521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6807d12b801447af87f2adae254975d9: Generated new TSK 0
I20260808 22:54:51.788849 2521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:54:51.825850 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:51.917259 2539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:51.917531 2539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:51.917577 2539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:51.918853 2539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:51.918929 2539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:51.918975 2539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:54:51.920619 2539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:51.920944 2539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:51.921206 2539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:51.926600 2545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:51.926601 2548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:51.926600 2546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:51.926838 2539 server_base.cc:1034] running on GCE node
I20260808 22:54:51.927131 2539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:51.927420 2539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:51.928609 2539 hybrid_clock.cc:648] HybridClock initialized: now 1786229691928599 us; error 49 us; skew 500 ppm
Aug 08 22:54:51 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:51.931468 2539 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:54:51.931921 2539 webserver.cc:460] Webserver started at http://127.1.212.1:39231/ using document root <none> and password file <none>
I20260808 22:54:51.932188 2539 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:51.932237 2539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:51.932336 2539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:51.933557 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "e5c6f699ed6247a680348b704b26cf63"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "47eef60d3c7a285029b10ba19b4c1f20"
server_key_iv: "441395b7e8d6477c134b69053c17312b"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.933970 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "e5c6f699ed6247a680348b704b26cf63"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "47eef60d3c7a285029b10ba19b4c1f20"
server_key_iv: "441395b7e8d6477c134b69053c17312b"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.935381 2539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:54:51.936146 2556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:51.936336 2539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:51.936455 2539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "e5c6f699ed6247a680348b704b26cf63"
format_stamp: "Formatted at 2026-08-08 22:54:51 on dist-test-slave-1x4g"
server_key: "47eef60d3c7a285029b10ba19b4c1f20"
server_key_iv: "441395b7e8d6477c134b69053c17312b"
server_key_version: "encryptionkey@0"
I20260808 22:54:51.936540 2539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:51.960173 2539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:51.962618 2539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:51.962762 2539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:51.963044 2539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:51.963428 2539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:51.963498 2539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:51.963537 2539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:51.963581 2539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:51.970827 2539 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:36481
I20260808 22:54:51.970881 2674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:36481 every 8 connection(s)
I20260808 22:54:51.971257 2539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:54:51.972810 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2539
I20260808 22:54:51.972918 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:54:51.973105 1872 external_mini_cluster.cc:1428] Setting key 6dc4dc271650027a039b218bb166350a
Aug 08 22:54:51 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:51.983539 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:51.972145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:59065 (local address 127.1.212.62:37121)
0808 22:54:51.972215 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:51.972218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:51.972414 (+ 196us) server_negotiation.cc:407] Connection header received
0808 22:54:51.973978 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:51.973980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:51.973993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:51.974041 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:51.974857 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.975377 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.976494 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.976753 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.976887 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:51.979905 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:51.979907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:51.979907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:51.979918 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:51.981918 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:51.982410 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:51.982412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:51.982412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:51.982479 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:51.982752 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:51.982753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:51.982754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:51.982928 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:51.983016 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:51.983317 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 22:54:51.983431 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:54:51.983589 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:51.972110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37121 (local address 127.1.212.1:59065)
0808 22:54:51.972282 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:51.972322 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:51.973856 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:51.974134 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:51.974138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:51.974428 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:51.974729 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:51.974735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.975490 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.975491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:51.976382 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:51.976386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:51.976905 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:51.976907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:51.977067 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:51.977855 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:51.977867 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:51.979749 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:51.982035 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:51.982044 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:51.982047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:51.982321 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:51.982556 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:51.982556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:51.982557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:51.982677 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:51.983005 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:51.983010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:51.983160 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 22:54:51.983256 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:54:51.983400 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
I20260808 22:54:51.984207 2676 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37121
I20260808 22:54:51.984318 2676 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:51.984549 2676 heartbeater.cc:511] Master 127.1.212.62:37121 requested a full tablet report, sending...
I20260808 22:54:51.985409 2473 ts_manager.cc:194] Registered new tserver with Master: e5c6f699ed6247a680348b704b26cf63 (127.1.212.1:36481)
I20260808 22:54:51.986070 2473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:59065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:54:52.004317 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:52.098528 2681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:52.098805 2681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:52.098850 2681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:52.100102 2681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:52.100183 2681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:52.100234 2681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:54:52.101917 2681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:52.102317 2681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:52.102615 2681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:52.108466 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:52.108515 2681 server_base.cc:1034] running on GCE node
W20260808 22:54:52.108489 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:52.108618 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:52.108946 2681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:52.109277 2681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:52.110466 2681 hybrid_clock.cc:648] HybridClock initialized: now 1786229692110437 us; error 42 us; skew 500 ppm
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:52.113549 2681 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:54:52.114091 2681 webserver.cc:460] Webserver started at http://127.1.212.2:38269/ using document root <none> and password file <none>
I20260808 22:54:52.114336 2681 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:52.114424 2681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:52.114584 2681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:52.115746 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "ffd9a194b5cd428fa4fc1b3001cd9333"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "589aa01d2cd5c6c93cf1e3977385d87d"
server_key_iv: "6ca32ba58fd1e8f87922065671f10b96"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.116145 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "ffd9a194b5cd428fa4fc1b3001cd9333"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "589aa01d2cd5c6c93cf1e3977385d87d"
server_key_iv: "6ca32ba58fd1e8f87922065671f10b96"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.117798 2681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:54:52.118755 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.118976 2681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:52.119076 2681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "ffd9a194b5cd428fa4fc1b3001cd9333"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "589aa01d2cd5c6c93cf1e3977385d87d"
server_key_iv: "6ca32ba58fd1e8f87922065671f10b96"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.119143 2681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:52.138284 2681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:52.140507 2681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:52.140637 2681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:52.141005 2681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:52.141315 2681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:52.141350 2681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.141376 2681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:52.141389 2681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.150091 2681 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:41865
I20260808 22:54:52.150131 2816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:41865 every 8 connection(s)
I20260808 22:54:52.150501 2681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:54:52.151175 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2681
I20260808 22:54:52.151280 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:54:52.151473 1872 external_mini_cluster.cc:1428] Setting key 72b08a3706ffece316dbc9bd59aff257
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
I20260808 22:54:52.163745 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.151382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51393 (local address 127.1.212.62:37121)
0808 22:54:52.151458 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.151461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.151671 (+ 210us) server_negotiation.cc:407] Connection header received
0808 22:54:52.153021 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.153023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.153037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.153086 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.154396 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.155159 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.156874 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.157159 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.157247 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.160194 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.160196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.160197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.160209 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.162088 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.162591 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.162593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.162594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.162660 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.162979 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.162981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.162981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.163118 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.163196 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.163529 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.163644 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:54:52.163977 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.151357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37121 (local address 127.1.212.2:51393)
0808 22:54:52.151546 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.151592 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.152889 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.153526 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.153531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.153956 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:52.154262 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.154269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.155288 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.155289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.156722 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.156728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.157291 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.157292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.157475 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.158087 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:52.158100 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:52.160064 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:52.162192 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:52.162206 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:52.162209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:52.162498 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:52.162738 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:52.162739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:52.162739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:52.162877 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:52.163199 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:52.163211 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:52.163384 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.163507 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.163702 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260808 22:54:52.164644 2817 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37121
I20260808 22:54:52.164820 2817 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:52.165141 2817 heartbeater.cc:511] Master 127.1.212.62:37121 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
I20260808 22:54:52.165683 2473 ts_manager.cc:194] Registered new tserver with Master: ffd9a194b5cd428fa4fc1b3001cd9333 (127.1.212.2:41865)
I20260808 22:54:52.166103 2473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:51393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:54:52.183739 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:52.274077 2823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:52.274333 2823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:52.274360 2823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:52.275569 2823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:52.275612 2823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:52.275640 2823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:54:52.277356 2823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:52.277592 2823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:52.277796 2823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:52.283365 2828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:52.283435 2830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:52.283365 2832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:52.283766 2823 server_base.cc:1034] running on GCE node
I20260808 22:54:52.284056 2823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:52.284369 2823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:52.285542 2823 hybrid_clock.cc:648] HybridClock initialized: now 1786229692285528 us; error 33 us; skew 500 ppm
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:52.288614 2823 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:54:52.289215 2823 webserver.cc:460] Webserver started at http://127.1.212.3:37117/ using document root <none> and password file <none>
I20260808 22:54:52.289497 2823 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:52.289546 2823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:52.289701 2823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:52.290895 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "798bc0a7b99c43c68ba143ffa9572cdc"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "58cac4a8652cf348bd97021bf049f62f"
server_key_iv: "4d68db4ad0c3fadb602800fe6066102d"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.291234 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "798bc0a7b99c43c68ba143ffa9572cdc"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "58cac4a8652cf348bd97021bf049f62f"
server_key_iv: "4d68db4ad0c3fadb602800fe6066102d"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.292740 2823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:54:52.293668 2839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.293882 2823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:52.293959 2823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "798bc0a7b99c43c68ba143ffa9572cdc"
format_stamp: "Formatted at 2026-08-08 22:54:52 on dist-test-slave-1x4g"
server_key: "58cac4a8652cf348bd97021bf049f62f"
server_key_iv: "4d68db4ad0c3fadb602800fe6066102d"
server_key_version: "encryptionkey@0"
I20260808 22:54:52.294027 2823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:52.303059 2823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:52.305867 2823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:52.305999 2823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:52.306299 2823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:54:52.306591 2823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:52.306625 2823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.306649 2823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:52.306663 2823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:52.316866 2823 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:45749
I20260808 22:54:52.316896 2959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:45749 every 8 connection(s)
I20260808 22:54:52.317335 2823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:54:52.319836 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 2823
I20260808 22:54:52.319945 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:54:52.320133 1872 external_mini_cluster.cc:1428] Setting key 72e0ee824f06d96297bd2831da63dc05
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:52.328030 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.318170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:55019 (local address 127.1.212.62:37121)
0808 22:54:52.318242 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.318244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.318463 (+ 219us) server_negotiation.cc:407] Connection header received
0808 22:54:52.319565 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.319567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.319579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.319700 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.320682 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.321397 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.322596 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.322785 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.322841 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.325437 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.325438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.325438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.325447 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.326739 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.327156 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.327157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.327158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.327218 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.327451 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.327451 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.327452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.327568 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.327624 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.327875 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.327959 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:54:52.328256 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.318135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37121 (local address 127.1.212.3:55019)
0808 22:54:52.318318 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.318365 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.319422 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.319781 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.319785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.320275 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:52.320602 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.320609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.321489 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.321490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.322514 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.322517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.322866 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.322867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.323012 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.323662 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:52.323671 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:52.325302 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:52.326820 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:52.326827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:52.326829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:52.327071 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:52.327282 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:52.327283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:52.327283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:52.327391 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:52.327626 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:52.327631 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:52.327762 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.327847 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.327979 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260808 22:54:52.328846 2960 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37121
I20260808 22:54:52.328979 2960 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:52.329198 2960 heartbeater.cc:511] Master 127.1.212.62:37121 requested a full tablet report, sending...
I20260808 22:54:52.329684 2473 ts_manager.cc:194] Registered new tserver with Master: 798bc0a7b99c43c68ba143ffa9572cdc (127.1.212.3:45749)
I20260808 22:54:52.330075 2473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:55019
I20260808 22:54:52.332130 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:54:52.340372 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.334062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.1.212.62:37121)
0808 22:54:52.334115 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.334117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.334192 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:54:52.334245 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.334246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.334255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.334302 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.334863 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.335350 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.336071 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.336261 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.336316 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.337737 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.337738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.337738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.337745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.339294 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.339618 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.339619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.339619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.339674 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.339828 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.339828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.339829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.339993 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.340069 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.340173 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.340239 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:54:52.342799 2473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:52.344758 2473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:52.352617 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.347739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36481 (local address 127.0.0.1:58164)
0808 22:54:52.348037 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.348050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.348117 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.348350 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.348354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.348356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:52.348445 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.348447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.350202 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.350203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.351281 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.351285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.352265 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.352266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.352352 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.352370 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.352453 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.352518 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260808 22:54:52.352727 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.348577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.1.212.3:45749)
0808 22:54:52.348996 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.349000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.349011 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:52.349082 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.349084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.349099 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.349186 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:52.349304 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.350367 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.351780 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.352334 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.352413 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.352478 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.352595 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.352640 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":97,"threads_started":1}
I20260808 22:54:52.352746 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.347969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:45749 (local address 127.0.0.1:39302)
0808 22:54:52.348197 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.348206 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.348243 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.349190 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.349191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.349193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:52.349230 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.349231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.350638 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.350639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.351686 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.351689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.352424 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.352425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.352505 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.352511 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.352579 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.352628 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":13,"thread_start_us":76,"threads_started":1}
I20260808 22:54:52.353510 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.347847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.1.212.1:36481)
0808 22:54:52.348046 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.348052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.348100 (+ 48us) server_negotiation.cc:407] Connection header received
0808 22:54:52.348200 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.348203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.348223 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.348330 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:52.348834 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.350073 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.351390 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.352167 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.352247 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.352305 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.353371 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.353434 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260808 22:54:52.353825 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.347737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41865 (local address 127.0.0.1:33672)
0808 22:54:52.347946 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.347992 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.348066 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.348616 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.348617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.348620 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:52.348725 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.348728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.351475 (+ 2747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.351476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.352534 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.352536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.353580 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.353582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.353648 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.353653 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.353708 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.353752 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260808 22:54:52.353869 2890 tablet_service.cc:1467] Processing CreateTablet for tablet 76ad26d172084d6f93789a558ee7ed22 (DEFAULT_TABLE table=test-table [id=6c49792b02fa478893a409a2460d47f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:52.354326 2890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ad26d172084d6f93789a558ee7ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:52.354336 2606 tablet_service.cc:1467] Processing CreateTablet for tablet 76ad26d172084d6f93789a558ee7ed22 (DEFAULT_TABLE table=test-table [id=6c49792b02fa478893a409a2460d47f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:52.354715 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ad26d172084d6f93789a558ee7ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:52.357515 2978 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Bootstrap starting.
I20260808 22:54:52.357924 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.348208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.1.212.2:41865)
0808 22:54:52.348428 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.348433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.348445 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:54:52.348490 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.348492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.348508 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.348602 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:52.350216 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.351370 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.352630 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.353489 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.353580 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.353638 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.357738 (+ 4100us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.357820 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":132,"threads_started":1}
I20260808 22:54:52.358140 2979 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Bootstrap starting.
I20260808 22:54:52.358321 2978 tablet_bootstrap.cc:654] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:52.358795 2748 tablet_service.cc:1467] Processing CreateTablet for tablet 76ad26d172084d6f93789a558ee7ed22 (DEFAULT_TABLE table=test-table [id=6c49792b02fa478893a409a2460d47f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:52.358974 2978 log.cc:824] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:52.359222 2748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ad26d172084d6f93789a558ee7ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:52.359865 2978 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: No bootstrap required, opened a new log
I20260808 22:54:52.359934 2978 ts_tablet_manager.cc:1397] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:54:52.361613 2978 raft_consensus.cc:348] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.361773 2978 raft_consensus.cc:374] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:52.361796 2978 raft_consensus.cc:729] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 798bc0a7b99c43c68ba143ffa9572cdc, State: Initialized, Role: FOLLOWER
I20260808 22:54:52.361936 2978 consensus_queue.cc:260] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.362277 2978 ts_tablet_manager.cc:1428] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 22:54:52.362318 2960 heartbeater.cc:503] Master 127.1.212.62:37121 was elected leader, sending a full tablet report...
I20260808 22:54:52.362476 2982 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Bootstrap starting.
I20260808 22:54:52.363315 2979 tablet_bootstrap.cc:654] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:52.363471 2982 tablet_bootstrap.cc:654] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:52.364131 2982 log.cc:824] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:52.364131 2979 log.cc:824] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:52.365119 2979 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: No bootstrap required, opened a new log
I20260808 22:54:52.365206 2979 ts_tablet_manager.cc:1397] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 22:54:52.365312 2982 tablet_bootstrap.cc:492] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: No bootstrap required, opened a new log
I20260808 22:54:52.365382 2982 ts_tablet_manager.cc:1397] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:54:52.367856 2982 raft_consensus.cc:348] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.367988 2979 raft_consensus.cc:348] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.368104 2982 raft_consensus.cc:374] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:52.368151 2979 raft_consensus.cc:374] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:52.368151 2982 raft_consensus.cc:729] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd9a194b5cd428fa4fc1b3001cd9333, State: Initialized, Role: FOLLOWER
I20260808 22:54:52.368185 2979 raft_consensus.cc:729] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c6f699ed6247a680348b704b26cf63, State: Initialized, Role: FOLLOWER
I20260808 22:54:52.368294 2979 consensus_queue.cc:260] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.368291 2982 consensus_queue.cc:260] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.368634 2979 ts_tablet_manager.cc:1428] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:54:52.368659 2817 heartbeater.cc:503] Master 127.1.212.62:37121 was elected leader, sending a full tablet report...
I20260808 22:54:52.368645 2982 ts_tablet_manager.cc:1428] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:54:52.368815 2676 heartbeater.cc:503] Master 127.1.212.62:37121 was elected leader, sending a full tablet report...
W20260808 22:54:52.401199 2819 tablet.cc:2369] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:54:52.472200 2677 tablet.cc:2369] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:54:52.517771 2981 raft_consensus.cc:482] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:52.517921 2981 raft_consensus.cc:504] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.518548 2981 leader_election.cc:288] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c6f699ed6247a680348b704b26cf63 (127.1.212.1:36481), ffd9a194b5cd428fa4fc1b3001cd9333 (127.1.212.2:41865)
I20260808 22:54:52.521965 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.518877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:42691 (local address 127.1.212.2:41865)
0808 22:54:52.518928 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.518930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.519001 (+ 71us) server_negotiation.cc:407] Connection header received
0808 22:54:52.519099 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.519100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.519117 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.519183 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:52.519387 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.519897 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521067 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.521544 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521648 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.521658 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.521827 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.521885 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:54:52.522010 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.518811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41865 (local address 127.1.212.3:42691)
0808 22:54:52.518888 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.518902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.519002 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.519207 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.519208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.519211 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:52.519305 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.519307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.520001 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.520002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.520974 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.520977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.521657 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.521745 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.521750 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.521822 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.521875 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":30}
I20260808 22:54:52.522210 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.518896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36481 (local address 127.1.212.3:47931)
0808 22:54:52.519097 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:52.519108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:52.519176 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:52.519311 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:52.519311 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:52.519313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:52.519408 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.519410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.520207 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.520208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.521234 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:52.521237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.521881 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:52.521958 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.521962 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:52.522063 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:54:52.522111 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":125,"threads_started":1}
I20260808 22:54:52.522260 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.519005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:47931 (local address 127.1.212.1:36481)
0808 22:54:52.519052 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.519054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.519188 (+ 134us) server_negotiation.cc:407] Connection header received
0808 22:54:52.519225 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.519226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.519238 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.519299 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:52.519527 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.520083 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521332 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.521796 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.521880 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.521891 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.522114 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.522168 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:54:52.522564 2768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ad26d172084d6f93789a558ee7ed22" candidate_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" is_pre_election: true
I20260808 22:54:52.522560 2628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ad26d172084d6f93789a558ee7ed22" candidate_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c6f699ed6247a680348b704b26cf63" is_pre_election: true
I20260808 22:54:52.522750 2628 raft_consensus.cc:2454] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 798bc0a7b99c43c68ba143ffa9572cdc in term 0.
I20260808 22:54:52.522750 2768 raft_consensus.cc:2454] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 798bc0a7b99c43c68ba143ffa9572cdc in term 0.
I20260808 22:54:52.523089 2842 leader_election.cc:302] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 798bc0a7b99c43c68ba143ffa9572cdc, e5c6f699ed6247a680348b704b26cf63; no voters:
I20260808 22:54:52.523243 2981 raft_consensus.cc:2781] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:52.523304 2981 raft_consensus.cc:482] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:52.523331 2981 raft_consensus.cc:3037] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:52.524205 2981 raft_consensus.cc:504] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.524370 2981 leader_election.cc:288] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [CANDIDATE]: Term 1 election: Requested vote from peers e5c6f699ed6247a680348b704b26cf63 (127.1.212.1:36481), ffd9a194b5cd428fa4fc1b3001cd9333 (127.1.212.2:41865)
I20260808 22:54:52.524552 2628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ad26d172084d6f93789a558ee7ed22" candidate_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c6f699ed6247a680348b704b26cf63"
I20260808 22:54:52.524552 2768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ad26d172084d6f93789a558ee7ed22" candidate_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333"
I20260808 22:54:52.524646 2628 raft_consensus.cc:3037] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:52.524646 2768 raft_consensus.cc:3037] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:52.525683 2628 raft_consensus.cc:2454] T 76ad26d172084d6f93789a558ee7ed22 P e5c6f699ed6247a680348b704b26cf63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 798bc0a7b99c43c68ba143ffa9572cdc in term 1.
I20260808 22:54:52.525683 2768 raft_consensus.cc:2454] T 76ad26d172084d6f93789a558ee7ed22 P ffd9a194b5cd428fa4fc1b3001cd9333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 798bc0a7b99c43c68ba143ffa9572cdc in term 1.
I20260808 22:54:52.525898 2841 leader_election.cc:302] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 798bc0a7b99c43c68ba143ffa9572cdc, ffd9a194b5cd428fa4fc1b3001cd9333; no voters:
I20260808 22:54:52.526054 2981 raft_consensus.cc:2781] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:52.526257 2981 raft_consensus.cc:686] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [term 1 LEADER]: Becoming Leader. State: Replica: 798bc0a7b99c43c68ba143ffa9572cdc, State: Running, Role: LEADER
I20260808 22:54:52.526371 2981 consensus_queue.cc:237] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } }
I20260808 22:54:52.527393 2471 catalog_manager.cc:5495] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 798bc0a7b99c43c68ba143ffa9572cdc (127.1.212.3). New cstate: current_term: 1 leader_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c6f699ed6247a680348b704b26cf63" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "798bc0a7b99c43c68ba143ffa9572cdc" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 45749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffd9a194b5cd428fa4fc1b3001cd9333" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41865 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:54:52.540463 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.531688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.1.212.1:36481)
0808 22:54:52.531741 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.531743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.531972 (+ 229us) server_negotiation.cc:407] Connection header received
0808 22:54:52.532062 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.532063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.532083 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.532151 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.532838 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.533347 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.534177 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.534378 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.534455 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.537121 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.537218 (+ 97us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.537219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.537235 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.539206 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.539587 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.539591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.539591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.539662 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.539870 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.539871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.539871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.540076 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.540192 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.540316 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.540392 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
W20260808 22:54:52.568050 2961 tablet.cc:2369] T 76ad26d172084d6f93789a558ee7ed22 P 798bc0a7b99c43c68ba143ffa9572cdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:54:52.568334 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.559898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.1.212.1:36481)
0808 22:54:52.560056 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.560058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.560074 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:54:52.560108 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.560109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.560120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.560199 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.560896 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.561439 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.562251 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.562450 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.562509 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.565226 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.565228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.565229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.565237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.566809 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.567256 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.567258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.567258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.567327 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.567549 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.567550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.567550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.567671 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.567780 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.568101 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.568243 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[2427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:54:52.596418 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:52.587828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.1.212.1:36481)
0808 22:54:52.587927 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:52.587930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:52.588025 (+ 95us) server_negotiation.cc:407] Connection header received
0808 22:54:52.588146 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:52.588147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:52.588159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:52.588264 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:52.589002 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.589595 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.590565 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:52.590788 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:52.590899 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:52.593518 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:52.593520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:52.593521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:52.593541 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:52.595022 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.595476 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.595478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.595479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.595552 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:52.595787 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:52.595788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:52.595789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:52.595914 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:52.596034 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:52.596183 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:54:52.596284 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 22:54:52.597201 2585 tablet_service.cc:2908] Rejecting scan request for tablet 76ad26d172084d6f93789a558ee7ed22: Uninitialized: clean time has not yet been initialized
I20260808 22:54:52.598368 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2539
I20260808 22:54:52.607061 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2681
I20260808 22:54:52.618072 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2823
I20260808 22:54:52.625705 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 2440
2026-08-08T22:54:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2252 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[3003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[3003](info): set up 2 sockets
Aug 08 22:54:52 dist-test-slave-1x4g krb5kdc[3003](info): commencing operation
krb5kdc: starting...
W20260808 22:54:53.712700 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:54:53 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:54:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:54:53.848285 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35351
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:35351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:53.935647 3018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:53.935869 3018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:53.935895 3018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:53.937150 3018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:54:53.937196 3018 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:54:53.937211 3018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:53.937250 3018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:53.937284 3018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:54:53.937301 3018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:54:53.938866 3018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:35351
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:53.939152 3018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:53.939409 3018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:53.944516 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:53.944541 3023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:53.944765 3018 server_base.cc:1034] running on GCE node
W20260808 22:54:53.944543 3024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:53.945163 3018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:53.945503 3018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:53.946674 3018 hybrid_clock.cc:648] HybridClock initialized: now 1786229693946651 us; error 36 us; skew 500 ppm
Aug 08 22:54:53 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:53.955340 3018 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:54:53.955871 3018 webserver.cc:460] Webserver started at http://127.1.212.62:41217/ using document root <none> and password file <none>
I20260808 22:54:53.956130 3018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:53.956202 3018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:53.956355 3018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:53.957594 3018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "d9205513b70d4c8196c9b8c8425f58e9"
format_stamp: "Formatted at 2026-08-08 22:54:53 on dist-test-slave-1x4g"
server_key: "eb9d44c2c35ce18f7b5df9b70920cd67"
server_key_iv: "69a16beb5ec9b51c4e5d451e99164481"
server_key_version: "encryptionkey@0"
I20260808 22:54:53.957993 3018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "d9205513b70d4c8196c9b8c8425f58e9"
format_stamp: "Formatted at 2026-08-08 22:54:53 on dist-test-slave-1x4g"
server_key: "eb9d44c2c35ce18f7b5df9b70920cd67"
server_key_iv: "69a16beb5ec9b51c4e5d451e99164481"
server_key_version: "encryptionkey@0"
I20260808 22:54:53.959463 3018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:54:53.960263 3033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:53.960529 3018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:53.960682 3018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "d9205513b70d4c8196c9b8c8425f58e9"
format_stamp: "Formatted at 2026-08-08 22:54:53 on dist-test-slave-1x4g"
server_key: "eb9d44c2c35ce18f7b5df9b70920cd67"
server_key_iv: "69a16beb5ec9b51c4e5d451e99164481"
server_key_version: "encryptionkey@0"
I20260808 22:54:53.960780 3018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:53.985077 3018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:53.987684 3018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:53.987845 3018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:53.993376 3018 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:35351
I20260808 22:54:53.993414 3100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:35351 every 8 connection(s)
I20260808 22:54:53.993826 3018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:54:53.994704 3101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:53.994850 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3018
I20260808 22:54:53.994956 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:54:53.995136 1872 external_mini_cluster.cc:1428] Setting key c1b76ee8e976cba55177d39d230ae74d
I20260808 22:54:53.998091 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Bootstrap starting.
I20260808 22:54:53.998775 3101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:53.999109 3101 log.cc:824] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:53.999867 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: No bootstrap required, opened a new log
Aug 08 22:54:53 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:54.002524 3101 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } }
I20260808 22:54:54.002686 3101 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:54.002709 3101 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9205513b70d4c8196c9b8c8425f58e9, State: Initialized, Role: FOLLOWER
I20260808 22:54:54.002867 3101 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } }
I20260808 22:54:54.002956 3101 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:54:54.003019 3101 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:54:54.003077 3101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:54.003804 3101 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } }
I20260808 22:54:54.003957 3101 leader_election.cc:302] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9205513b70d4c8196c9b8c8425f58e9; no voters:
I20260808 22:54:54.004201 3101 leader_election.cc:288] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:54:54.004235 3106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:54.004359 3106 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [term 1 LEADER]: Becoming Leader. State: Replica: d9205513b70d4c8196c9b8c8425f58e9, State: Running, Role: LEADER
I20260808 22:54:54.004523 3101 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:54:54.004529 3106 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } }
I20260808 22:54:54.005290 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9205513b70d4c8196c9b8c8425f58e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } } }
I20260808 22:54:54.005478 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:54.005309 3108 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9205513b70d4c8196c9b8c8425f58e9. Latest consensus state: current_term: 1 leader_uuid: "d9205513b70d4c8196c9b8c8425f58e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9205513b70d4c8196c9b8c8425f58e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35351 } } }
I20260808 22:54:54.005792 3108 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:54.005961 3116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:54:54.006670 3116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:54:54.006963 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:53.995721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.1.212.62:35351)
0808 22:54:53.995932 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:53.995942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:53.995970 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:54:53.996423 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:53.996430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:53.996493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:53.996663 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:53.997370 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:53.998350 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:53.999199 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:53.999635 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:53.999762 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.002619 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.002623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.002624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.002637 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.004808 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.005233 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.005238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.005238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.005353 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.005673 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.005675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.005675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.006075 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.006303 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.006569 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.006730 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260808 22:54:54.008528 3116 catalog_manager.cc:1347] Generated new cluster ID: 321c094cc62a40858613e283b97f0858
I20260808 22:54:54.008592 3116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:54:54.019085 3116 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:54:54.019930 3116 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:54:54.028769 3116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Generated new TSK 0
I20260808 22:54:54.029054 3116 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:54:54.061483 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35351
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:54.155987 3127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:54.156246 3127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:54.156275 3127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:54.157707 3127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:54.157763 3127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:54.157786 3127 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:54.157805 3127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:54:54.159397 3127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:35351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:54.159634 3127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:54.159840 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:54.165673 3133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:54.165725 3136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:54.165755 3134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:54.165993 3127 server_base.cc:1034] running on GCE node
I20260808 22:54:54.166236 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:54.166518 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:54.167701 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786229694167692 us; error 54 us; skew 500 ppm
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:54.170755 3127 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:54:54.171316 3127 webserver.cc:460] Webserver started at http://127.1.212.1:42745/ using document root <none> and password file <none>
I20260808 22:54:54.171586 3127 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:54.171669 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:54.171839 3127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:54.173167 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "f218d36650f64ff5a2cd46ee04f17476"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "a059f3ae413014ae1fdfb0e73e5a3b08"
server_key_iv: "54a3c32d0280e725d78d2ef14aae496a"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.173601 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "f218d36650f64ff5a2cd46ee04f17476"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "a059f3ae413014ae1fdfb0e73e5a3b08"
server_key_iv: "54a3c32d0280e725d78d2ef14aae496a"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.175158 3127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 22:54:54.176141 3143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.176380 3127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:54.176512 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "f218d36650f64ff5a2cd46ee04f17476"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "a059f3ae413014ae1fdfb0e73e5a3b08"
server_key_iv: "54a3c32d0280e725d78d2ef14aae496a"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.176618 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:54.210402 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:54.212991 3127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:54.213194 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:54.213927 3127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:54.213994 3127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.214058 3127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:54.214092 3127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:54.226943 3127 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:39201
I20260808 22:54:54.227142 3265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:39201 every 8 connection(s)
I20260808 22:54:54.227485 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:54:54.227530 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.214877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:41323 (local address 127.1.212.62:35351)
0808 22:54:54.214964 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.214968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.215226 (+ 258us) server_negotiation.cc:407] Connection header received
0808 22:54:54.215775 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.215778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.215793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.215902 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.216918 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.217500 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.219462 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.219804 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.219894 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.223484 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.223496 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.223496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.223523 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.225645 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.226201 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.226202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.226202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.226279 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.226581 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.226582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.226583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.226729 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.226817 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.227252 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.227368 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:54:54.227663 3153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.214829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.1:41323)
0808 22:54:54.215049 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.215096 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.215640 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.216013 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.216017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.216291 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.216741 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.216748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.217633 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.217634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.219330 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.219336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.219923 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.219925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.220127 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.220833 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.220845 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.223287 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.225779 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.225790 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.225794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.226089 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.226364 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.226365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.226365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.226500 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.226807 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.226815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.227050 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.227209 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.227409 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":139,"threads_started":1}
I20260808 22:54:54.228530 3266 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35351
I20260808 22:54:54.228641 3266 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:54.228601 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3127
I20260808 22:54:54.228732 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:54:54.228878 3266 heartbeater.cc:511] Master 127.1.212.62:35351 requested a full tablet report, sending...
I20260808 22:54:54.228976 1872 external_mini_cluster.cc:1428] Setting key 8a73d9846b1a3e8435f59acd14701122
I20260808 22:54:54.229486 3050 ts_manager.cc:194] Registered new tserver with Master: f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1:39201)
I20260808 22:54:54.230399 3050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:41323
I20260808 22:54:54.239018 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.231512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.1:53249)
0808 22:54:54.231777 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.231799 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.231929 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.232136 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.232137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.232334 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.232435 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.232437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.233481 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.233483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.234013 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.234015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.234517 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.234518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.234581 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.235230 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.235235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.235565 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.237728 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.237729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.237730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.237945 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.238188 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.238188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.238189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.238257 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.238634 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.238635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.238701 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.238786 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.238917 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":166,"threads_started":1}
I20260808 22:54:54.239040 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.231594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:53249 (local address 127.1.212.62:35351)
0808 22:54:54.231639 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.231641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.231882 (+ 241us) server_negotiation.cc:407] Connection header received
0808 22:54:54.232003 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.232004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.232016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.232083 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.232531 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.233331 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.234111 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.234431 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.234511 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.235667 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.235669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.235670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.235679 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.237609 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.238035 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.238036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.238037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.238109 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.238329 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.238330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.238330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.238494 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.238583 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.238830 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.238931 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:54:54.260203 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35351
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:54.351481 3275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:54.351723 3275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:54.351752 3275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:54.353098 3275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:54.353150 3275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:54.353173 3275 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:54.353221 3275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:54:54.354838 3275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:35351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:54.355147 3275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:54.355418 3275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:54.360894 3282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:54.360893 3281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:54.360910 3284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:54.361236 3275 server_base.cc:1034] running on GCE node
I20260808 22:54:54.361518 3275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:54.361824 3275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:54.363020 3275 hybrid_clock.cc:648] HybridClock initialized: now 1786229694363006 us; error 29 us; skew 500 ppm
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:54.366216 3275 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:54:54.366796 3275 webserver.cc:460] Webserver started at http://127.1.212.2:46741/ using document root <none> and password file <none>
I20260808 22:54:54.367092 3275 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:54.367146 3275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:54.367326 3275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:54.368764 3275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "c219788fb9ed452d8228824d02688db4"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "5b79a2cce26d7139412ec35ebd077bdf"
server_key_iv: "bdec7bfe2b178fb5c2da64776f3c1daf"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.369277 3275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "c219788fb9ed452d8228824d02688db4"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "5b79a2cce26d7139412ec35ebd077bdf"
server_key_iv: "bdec7bfe2b178fb5c2da64776f3c1daf"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.370836 3275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:54:54.371815 3291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.372037 3275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:54.372135 3275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "c219788fb9ed452d8228824d02688db4"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "5b79a2cce26d7139412ec35ebd077bdf"
server_key_iv: "bdec7bfe2b178fb5c2da64776f3c1daf"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.372221 3275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:54.388319 3275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:54.390743 3275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:54.390877 3275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:54.391673 3275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:54.391731 3275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.391767 3275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:54.391786 3275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:54.402784 3275 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:38727
I20260808 22:54:54.402959 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:38727 every 8 connection(s)
I20260808 22:54:54.403321 3275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:54:54.407238 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3275
I20260808 22:54:54.407294 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.392191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:50039 (local address 127.1.212.62:35351)
0808 22:54:54.392271 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.392274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.392633 (+ 359us) server_negotiation.cc:407] Connection header received
0808 22:54:54.393257 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.393259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.393276 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.393369 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.394223 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.395188 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.397338 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.397851 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.397964 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.402883 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.402885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.402885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.402895 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.404882 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.405921 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.405923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.405923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.406013 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.406470 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.406471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.406471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.406604 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.406660 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.407078 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.407183 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:54:54.407351 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:54:54.407438 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.392160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.2:50039)
0808 22:54:54.392339 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.392386 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.393139 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.393485 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.393490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.393839 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.394099 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.394105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.395298 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.395299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.397199 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.397204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.397980 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.397982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.398200 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.399036 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.399052 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.402686 (+ 3634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.405110 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.405208 (+ 98us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.405213 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.405800 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.406139 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.406140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.406141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.406311 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.406691 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.406699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.406880 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.406981 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.407162 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260808 22:54:54.407565 1872 external_mini_cluster.cc:1428] Setting key 715388e6c8475b136b04e974972d51f5
I20260808 22:54:54.408123 3411 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35351
I20260808 22:54:54.408221 3411 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:54.408444 3411 heartbeater.cc:511] Master 127.1.212.62:35351 requested a full tablet report, sending...
I20260808 22:54:54.408905 3049 ts_manager.cc:194] Registered new tserver with Master: c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727)
I20260808 22:54:54.409478 3049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:50039
I20260808 22:54:54.417522 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.409646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46651 (local address 127.1.212.62:35351)
0808 22:54:54.410451 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.410453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.410789 (+ 336us) server_negotiation.cc:407] Connection header received
0808 22:54:54.410852 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.410853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.410862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.410909 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.411646 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.412154 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.413101 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.413311 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.413402 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.414644 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.414646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.414646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.414654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.416134 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.416500 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.416501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.416501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.416566 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.416858 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.416859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.416859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.417042 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.417146 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.417325 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.417421 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771}
I20260808 22:54:54.417526 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.409694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.2:46651)
0808 22:54:54.410697 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.410713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.410790 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.410928 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.410929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.411445 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.411530 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.411533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.412324 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.412325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.412992 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.412995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.413405 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.413406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.413526 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.414181 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.414186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.414531 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.416229 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.416230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.416231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.416421 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.416672 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.416673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.416674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.416744 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.417141 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.417142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.417199 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.417284 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.417417 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:54:54.444465 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35351
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:54.538132 3422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:54.538383 3422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:54.538416 3422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:54.539671 3422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:54.539718 3422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:54.539740 3422 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:54.539759 3422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:54:54.541631 3422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:35351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:54.541867 3422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:54.542078 3422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:54.547602 3428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:54.547571 3430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:54.547693 3422 server_base.cc:1034] running on GCE node
W20260808 22:54:54.547572 3427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:54.548208 3422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:54.548501 3422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:54.549674 3422 hybrid_clock.cc:648] HybridClock initialized: now 1786229694549668 us; error 39 us; skew 500 ppm
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:54.552393 3422 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:54:54.552963 3422 webserver.cc:460] Webserver started at http://127.1.212.3:44109/ using document root <none> and password file <none>
I20260808 22:54:54.553223 3422 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:54.553294 3422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:54.553457 3422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:54.554713 3422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "b1dd46becb5e68341d89e787be7280be"
server_key_iv: "0650c393bf047bbc1200c5407795c3dc"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.555120 3422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "b1dd46becb5e68341d89e787be7280be"
server_key_iv: "0650c393bf047bbc1200c5407795c3dc"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.556645 3422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:54:54.557559 3437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.557765 3422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:54.557881 3422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
format_stamp: "Formatted at 2026-08-08 22:54:54 on dist-test-slave-1x4g"
server_key: "b1dd46becb5e68341d89e787be7280be"
server_key_iv: "0650c393bf047bbc1200c5407795c3dc"
server_key_version: "encryptionkey@0"
I20260808 22:54:54.557969 3422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:54.566817 3422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:54.569517 3422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:54.569800 3422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:54.570377 3422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:54.570446 3422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:54.570497 3422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:54.570552 3422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:54.579285 3422 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:44357
I20260808 22:54:54.579438 3561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:44357 every 8 connection(s)
I20260808 22:54:54.579803 3422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:54:54.581372 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3422
I20260808 22:54:54.581477 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:54:54.581663 1872 external_mini_cluster.cc:1428] Setting key 9bf76c94e174421e37a3cdad9458aa94
I20260808 22:54:54.582929 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.571188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:40607 (local address 127.1.212.62:35351)
0808 22:54:54.571254 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.571256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.571457 (+ 201us) server_negotiation.cc:407] Connection header received
0808 22:54:54.572095 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.572097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.572112 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.572176 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.573214 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.573747 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.574848 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.575173 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.575254 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.579256 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.579258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.579258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.579270 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.581124 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.581607 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.581609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.581609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.581688 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.582023 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.582024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.582024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.582226 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.582281 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.582735 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.582835 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:54:54.583151 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.571099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.3:40607)
0808 22:54:54.571308 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.571360 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.571968 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.572274 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.572278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.572680 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.573097 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.573103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.573853 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.573854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.574745 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.574748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.575284 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.575285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.575503 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.576198 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.576212 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.579139 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.581244 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.581251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.581253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.581529 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.581789 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.581790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.581790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.581938 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.582334 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.582341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.582545 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.582697 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.582897 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260808 22:54:54.583681 3562 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35351
I20260808 22:54:54.583798 3562 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:54.584002 3562 heartbeater.cc:511] Master 127.1.212.62:35351 requested a full tablet report, sending...
I20260808 22:54:54.584491 3050 ts_manager.cc:194] Registered new tserver with Master: 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:54.585033 3050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:40607
I20260808 22:54:54.590837 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.585080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:35593 (local address 127.1.212.62:35351)
0808 22:54:54.585130 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.585132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.585275 (+ 143us) server_negotiation.cc:407] Connection header received
0808 22:54:54.585311 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.585312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.585323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.585393 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.585823 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.586360 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.587132 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.587317 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.587370 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.588368 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.588369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.588370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.588376 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.589847 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.590193 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.590194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.590194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.590248 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.590417 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.590418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.590418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.590522 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.590572 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.590707 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.590772 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:54:54.590850 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.585002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.1.212.3:35593)
0808 22:54:54.585193 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.585205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.585267 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.585398 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.585398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.585673 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:54.585738 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.585740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.586493 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.586494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.587042 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.587043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.587405 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.587406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.587492 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.587952 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:54.587957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:54.588280 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:54.589930 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.589931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.589931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.590122 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.590308 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:54.590309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:54.590309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:54.590364 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:54.590575 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:54.590576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:54.590618 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.590672 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.590764 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260808 22:54:54.593703 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:54:54 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:54.619323 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.611432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.1.212.62:35351)
0808 22:54:54.611505 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.611507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.611516 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:54:54.611544 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.611545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.611553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.611614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:54.612203 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.612715 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.613598 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.613784 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.613842 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.616404 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:54.616406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:54.616407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:54.616422 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:54.618016 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.618462 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.618464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.618464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.618531 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:54.618743 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:54.618744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:54.618744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:54.618872 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:54.618985 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.619117 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.619211 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:54:54.621286 3050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:54.622998 3050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:54.631348 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38727 (local address 127.0.0.1:51386)
0808 22:54:54.626666 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.626726 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.626811 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.627397 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.627401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.627404 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:54.627498 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.627500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.628657 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.628658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.630123 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.630126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.631060 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.631061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.631136 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631141 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.631194 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.631244 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":178,"threads_started":1}
I20260808 22:54:54.631441 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.1.212.3:44357)
0808 22:54:54.627717 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.627723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.627733 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:54:54.627774 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.627777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.627794 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.627908 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:54.628057 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.629150 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630446 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.631051 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.631139 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631199 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.631284 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.631331 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":112,"threads_started":1}
I20260808 22:54:54.631349 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:44357 (local address 127.0.0.1:56394)
0808 22:54:54.626621 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.626777 (+ 156us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.626853 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.627907 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.627908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.627911 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:54.627961 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.627962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.629255 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.629256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.630351 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.630353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.631139 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.631140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.631203 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631207 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.631264 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.631295 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":81,"threads_started":1}
I20260808 22:54:54.631700 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.1.212.1:39201)
0808 22:54:54.627383 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.627388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.627399 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:54.627436 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.627438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.627453 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.627563 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:54.627738 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.628958 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630046 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.630834 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630930 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631003 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.631517 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.631576 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":120,"threads_started":1}
I20260808 22:54:54.631866 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39201 (local address 127.0.0.1:50340)
0808 22:54:54.627286 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.627299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.627345 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.627577 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.627578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.627581 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:54.627644 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.627646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.629068 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.629069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.629923 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.629926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.630943 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.631035 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631052 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.631126 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.631189 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":799,"thread_start_us":89,"threads_started":1}
I20260808 22:54:54.632117 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.626996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.1.212.2:38727)
0808 22:54:54.627184 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.627190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.627201 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:54.627245 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.627247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.627267 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.627367 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:54.627577 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.628525 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630220 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.630914 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.630977 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.631036 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.631874 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.632014 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260808 22:54:54.632443 3491 tablet_service.cc:1467] Processing CreateTablet for tablet 194adc040c534436a33954a024a72178 (DEFAULT_TABLE table=test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:54.632567 3189 tablet_service.cc:1467] Processing CreateTablet for tablet 194adc040c534436a33954a024a72178 (DEFAULT_TABLE table=test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:54.632949 3491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194adc040c534436a33954a024a72178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:54.632843 3342 tablet_service.cc:1467] Processing CreateTablet for tablet 194adc040c534436a33954a024a72178 (DEFAULT_TABLE table=test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:54.632970 3189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194adc040c534436a33954a024a72178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:54.633175 3342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194adc040c534436a33954a024a72178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:54.635968 3587 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Bootstrap starting.
I20260808 22:54:54.636881 3589 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Bootstrap starting.
I20260808 22:54:54.637176 3587 tablet_bootstrap.cc:654] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:54.637583 3587 log.cc:824] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:54.637611 3589 tablet_bootstrap.cc:654] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:54.637965 3589 log.cc:824] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:54.638198 3590 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Bootstrap starting.
I20260808 22:54:54.638502 3587 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: No bootstrap required, opened a new log
I20260808 22:54:54.638588 3587 ts_tablet_manager.cc:1397] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:54:54.638800 3589 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: No bootstrap required, opened a new log
I20260808 22:54:54.638872 3589 ts_tablet_manager.cc:1397] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:54:54.638990 3590 tablet_bootstrap.cc:654] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:54.639323 3590 log.cc:824] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:54.640024 3590 tablet_bootstrap.cc:492] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: No bootstrap required, opened a new log
I20260808 22:54:54.640110 3590 ts_tablet_manager.cc:1397] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:54:54.640758 3587 raft_consensus.cc:348] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.640993 3587 raft_consensus.cc:374] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:54.640898 3589 raft_consensus.cc:348] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.641038 3589 raft_consensus.cc:374] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:54.641033 3587 raft_consensus.cc:729] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd06e8ee0747ef92d9e9cce7377545, State: Initialized, Role: FOLLOWER
I20260808 22:54:54.641078 3589 raft_consensus.cc:729] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c219788fb9ed452d8228824d02688db4, State: Initialized, Role: FOLLOWER
I20260808 22:54:54.641185 3587 consensus_queue.cc:260] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.641185 3589 consensus_queue.cc:260] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.641485 3589 ts_tablet_manager.cc:1428] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:54:54.641489 3587 ts_tablet_manager.cc:1428] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 22:54:54.641614 3562 heartbeater.cc:503] Master 127.1.212.62:35351 was elected leader, sending a full tablet report...
I20260808 22:54:54.641659 3411 heartbeater.cc:503] Master 127.1.212.62:35351 was elected leader, sending a full tablet report...
I20260808 22:54:54.642131 3590 raft_consensus.cc:348] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.642303 3590 raft_consensus.cc:374] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:54.642339 3590 raft_consensus.cc:729] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f218d36650f64ff5a2cd46ee04f17476, State: Initialized, Role: FOLLOWER
I20260808 22:54:54.642453 3590 consensus_queue.cc:260] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.642753 3590 ts_tablet_manager.cc:1428] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:54:54.642871 3266 heartbeater.cc:503] Master 127.1.212.62:35351 was elected leader, sending a full tablet report...
W20260808 22:54:54.654222 3412 tablet.cc:2369] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:54:54.729656 3267 tablet.cc:2369] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:54:54.830557 3563 tablet.cc:2369] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:54:54.902103 3594 raft_consensus.cc:482] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:54.902298 3594 raft_consensus.cc:504] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.902789 3594 leader_election.cc:288] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1:39201), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:54.906416 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.902981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39201 (local address 127.1.212.2:34883)
0808 22:54:54.903440 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.903458 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.903546 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.903678 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.903679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.903681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:54.903771 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.903772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.904626 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.904627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.905517 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.905520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.906203 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.906203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.906265 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.906269 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.906309 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.906341 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":301,"thread_start_us":90,"threads_started":1}
I20260808 22:54:54.906448 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.903048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:34883 (local address 127.1.212.1:39201)
0808 22:54:54.903267 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.903270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.903551 (+ 281us) server_negotiation.cc:407] Connection header received
0808 22:54:54.903603 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.903604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.903615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.903670 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:54.903850 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.904531 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.905649 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.906122 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.906189 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.906202 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.906334 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.906377 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260808 22:54:54.906929 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.902981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:44357 (local address 127.1.212.2:54785)
0808 22:54:54.903079 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:54.903101 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:54.903149 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:54.903391 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:54.903392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:54.903394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:54.903489 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.903490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.904526 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.904527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.905904 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:54.905907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.906699 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.906700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:54.906779 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.906782 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:54:54.906836 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:54:54.906878 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 22:54:54.906880 3216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194adc040c534436a33954a024a72178" candidate_uuid: "c219788fb9ed452d8228824d02688db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f218d36650f64ff5a2cd46ee04f17476" is_pre_election: true
I20260808 22:54:54.907078 3216 raft_consensus.cc:2454] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c219788fb9ed452d8228824d02688db4 in term 0.
I20260808 22:54:54.907171 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.903160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:54785 (local address 127.1.212.3:44357)
0808 22:54:54.903237 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.903239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.903250 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:54.903287 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.903288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.903301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.903573 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:54.903627 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.904394 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.906025 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.906611 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.906690 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.906702 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.907050 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.907105 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:54:54.907357 3294 leader_election.cc:302] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c219788fb9ed452d8228824d02688db4, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:54.907485 3594 raft_consensus.cc:2781] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:54.907438 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194adc040c534436a33954a024a72178" candidate_uuid: "c219788fb9ed452d8228824d02688db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" is_pre_election: true
I20260808 22:54:54.907545 3594 raft_consensus.cc:482] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:54.907606 3594 raft_consensus.cc:3037] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:54.907608 3509 raft_consensus.cc:2454] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c219788fb9ed452d8228824d02688db4 in term 0.
I20260808 22:54:54.908861 3594 raft_consensus.cc:504] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.909034 3594 leader_election.cc:288] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [CANDIDATE]: Term 1 election: Requested vote from peers f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1:39201), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:54.909198 3216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194adc040c534436a33954a024a72178" candidate_uuid: "c219788fb9ed452d8228824d02688db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f218d36650f64ff5a2cd46ee04f17476"
I20260808 22:54:54.909243 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194adc040c534436a33954a024a72178" candidate_uuid: "c219788fb9ed452d8228824d02688db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
I20260808 22:54:54.909283 3216 raft_consensus.cc:3037] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:54.909337 3509 raft_consensus.cc:3037] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:54.910028 3216 raft_consensus.cc:2454] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c219788fb9ed452d8228824d02688db4 in term 1.
I20260808 22:54:54.910161 3509 raft_consensus.cc:2454] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c219788fb9ed452d8228824d02688db4 in term 1.
I20260808 22:54:54.910219 3294 leader_election.cc:302] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c219788fb9ed452d8228824d02688db4, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:54.910406 3594 raft_consensus.cc:2781] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:54.910634 3594 raft_consensus.cc:686] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 1 LEADER]: Becoming Leader. State: Replica: c219788fb9ed452d8228824d02688db4, State: Running, Role: LEADER
I20260808 22:54:54.910769 3594 consensus_queue.cc:237] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:54.911705 3049 catalog_manager.cc:5495] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c219788fb9ed452d8228824d02688db4 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "c219788fb9ed452d8228824d02688db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } health_report { overall_health: UNKNOWN } } }
I20260808 22:54:54.986563 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:54.984094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51402 (local address 127.1.212.2:38727)
0808 22:54:54.984144 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:54.984146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:54.984175 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:54:54.984268 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:54.984269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:54.984290 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:54.984348 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:54.984535 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.985041 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.985819 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:54.986010 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:54.986062 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:54.986276 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:54.986374 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:54.986432 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:54.986481 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 22:54:54.986500 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:54:54.988467 3216 raft_consensus.cc:1261] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Refusing update from remote peer c219788fb9ed452d8228824d02688db4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:54.988478 3509 raft_consensus.cc:1261] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Refusing update from remote peer c219788fb9ed452d8228824d02688db4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:54.988904 3598 consensus_queue.cc:1035] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:54.989121 3594 consensus_queue.cc:1035] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:54.991617 3601 mvcc.cc:205] Tried to move back new op lower bound from 7316396830669856768 to 7316396830355316736. Current Snapshot: MvccSnapshot[applied={T|T < 7316396830669856768}]
I20260808 22:54:54.991675 3603 mvcc.cc:205] Tried to move back new op lower bound from 7316396830669856768 to 7316396830355316736. Current Snapshot: MvccSnapshot[applied={T|T < 7316396830669856768}]
I20260808 22:54:54.996428 3050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:54.996560 3050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:54.997524 3050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Sending DeleteTablet for 3 replicas of tablet 194adc040c534436a33954a024a72178
I20260808 22:54:54.997903 3189 tablet_service.cc:1514] Processing DeleteTablet for tablet 194adc040c534436a33954a024a72178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 22:54:54.997960 3491 tablet_service.cc:1514] Processing DeleteTablet for tablet 194adc040c534436a33954a024a72178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:56394
I20260808 22:54:54.998400 3342 tablet_service.cc:1514] Processing DeleteTablet for tablet 194adc040c534436a33954a024a72178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:51386
I20260808 22:54:54.998566 3612 tablet_replica.cc:330] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: stopping tablet replica
I20260808 22:54:54.998706 3613 tablet_replica.cc:330] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: stopping tablet replica
I20260808 22:54:54.998708 3612 raft_consensus.cc:2229] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:54.998718 3050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:54:54.998847 3612 raft_consensus.cc:2258] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 22:54:54.998894 3050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:54.999089 3613 raft_consensus.cc:2229] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:54:54.999267 3613 raft_consensus.cc:2258] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:54.999406 3612 ts_tablet_manager.cc:1905] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.000030 3613 ts_tablet_manager.cc:1905] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.000762 3491 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba (DEFAULT_TABLE table=test-table [id=db9c541bb1e5467dae3ebae800d6901e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:55.000933 3491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31c68ad70943d38b2c7b691df9bdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.001498 3612 ts_tablet_manager.cc:1918] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:54:55.001598 3612 log.cc:1196] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/194adc040c534436a33954a024a72178
I20260808 22:54:55.001888 3612 ts_tablet_manager.cc:1939] T 194adc040c534436a33954a024a72178 P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting consensus metadata
I20260808 22:54:55.002276 3587 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Bootstrap starting.
I20260808 22:54:55.002391 3613 ts_tablet_manager.cc:1918] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:54:55.002475 3613 log.cc:1196] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/194adc040c534436a33954a024a72178
I20260808 22:54:55.002727 3613 ts_tablet_manager.cc:1939] T 194adc040c534436a33954a024a72178 P c219788fb9ed452d8228824d02688db4: Deleting consensus metadata
I20260808 22:54:55.002724 3587 tablet_bootstrap.cc:654] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.003285 3037 catalog_manager.cc:4841] TS c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727): tablet 194adc040c534436a33954a024a72178 (table test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]) successfully deleted
I20260808 22:54:55.003432 3587 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: No bootstrap required, opened a new log
I20260808 22:54:55.003463 3035 catalog_manager.cc:4841] TS 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357): tablet 194adc040c534436a33954a024a72178 (table test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]) successfully deleted
I20260808 22:54:55.003489 3587 ts_tablet_manager.cc:1397] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:54:55.003654 3587 raft_consensus.cc:348] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.003682 3189 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba (DEFAULT_TABLE table=test-table [id=db9c541bb1e5467dae3ebae800d6901e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:55.003741 3587 raft_consensus.cc:374] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.003775 3587 raft_consensus.cc:729] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd06e8ee0747ef92d9e9cce7377545, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.003815 3189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31c68ad70943d38b2c7b691df9bdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.003820 3587 consensus_queue.cc:260] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.003919 3587 ts_tablet_manager.cc:1428] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:55.004944 3342 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba (DEFAULT_TABLE table=test-table [id=db9c541bb1e5467dae3ebae800d6901e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:55.005087 3342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31c68ad70943d38b2c7b691df9bdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.005386 3590 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Bootstrap starting.
I20260808 22:54:55.005817 3590 tablet_bootstrap.cc:654] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.006618 3590 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: No bootstrap required, opened a new log
I20260808 22:54:55.006661 3589 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Bootstrap starting.
I20260808 22:54:55.006673 3590 ts_tablet_manager.cc:1397] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:55.006837 3611 tablet_replica.cc:330] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: stopping tablet replica
I20260808 22:54:55.006845 3590 raft_consensus.cc:348] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.006987 3611 raft_consensus.cc:2229] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:55.007012 3590 raft_consensus.cc:374] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.007035 3590 raft_consensus.cc:729] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f218d36650f64ff5a2cd46ee04f17476, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.007076 3590 consensus_queue.cc:260] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.007121 3611 raft_consensus.cc:2258] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:55.007170 3589 tablet_bootstrap.cc:654] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.007177 3590 ts_tablet_manager.cc:1428] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:55.007741 3611 ts_tablet_manager.cc:1905] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.008402 3589 tablet_bootstrap.cc:492] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: No bootstrap required, opened a new log
I20260808 22:54:55.008472 3589 ts_tablet_manager.cc:1397] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:54:55.008627 3589 raft_consensus.cc:348] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.008723 3589 raft_consensus.cc:374] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.008745 3589 raft_consensus.cc:729] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c219788fb9ed452d8228824d02688db4, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.008809 3589 consensus_queue.cc:260] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.008921 3589 ts_tablet_manager.cc:1428] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:55.009521 3611 ts_tablet_manager.cc:1918] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:54:55.009651 3611 log.cc:1196] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/194adc040c534436a33954a024a72178
I20260808 22:54:55.010171 3611 ts_tablet_manager.cc:1939] T 194adc040c534436a33954a024a72178 P f218d36650f64ff5a2cd46ee04f17476: Deleting consensus metadata
I20260808 22:54:55.010656 3036 catalog_manager.cc:4841] TS f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1:39201): tablet 194adc040c534436a33954a024a72178 (table test-table [id=eff8ccc4509948cd9e53c06ebbdd9b35]) successfully deleted
I20260808 22:54:55.078609 3596 raft_consensus.cc:482] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:55.078722 3596 raft_consensus.cc:504] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.079119 3596 leader_election.cc:288] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:55.083082 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.079397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:45061 (local address 127.1.212.2:38727)
0808 22:54:55.079460 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.079462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.079558 (+ 96us) server_negotiation.cc:407] Connection header received
0808 22:54:55.079681 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.079682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.079691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.079726 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:55.080030 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.080852 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.082280 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.082648 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.082738 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.082750 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.082964 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.083009 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:54:55.083366 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.079327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38727 (local address 127.1.212.1:45061)
0808 22:54:55.079463 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:55.079475 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:55.079583 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:55.079802 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:55.079803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:55.079805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:55.079929 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.079932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.081072 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.081073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.082161 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.082163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.082755 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.082756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.082840 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.082844 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:55.082928 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:54:55.082980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":36,"thread_start_us":63,"threads_started":1}
I20260808 22:54:55.083770 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.079333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:44357 (local address 127.1.212.1:56585)
0808 22:54:55.079503 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:55.079512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:55.079566 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:55.079802 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:55.079803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:55.079806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:55.079929 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.079932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.080708 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.080709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.082143 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.082147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.083080 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.083082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.083554 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.083561 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:54:55.083629 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:54:55.083678 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":77,"thread_start_us":81,"threads_started":1}
I20260808 22:54:55.083796 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.079541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56585 (local address 127.1.212.3:44357)
0808 22:54:55.079592 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.079594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.079605 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:55.079677 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.079679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.079691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.079726 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:55.080033 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.080589 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.082268 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.082831 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.082918 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.082930 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.083658 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.083718 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:54:55.083854 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31c68ad70943d38b2c7b691df9bdba" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c219788fb9ed452d8228824d02688db4" is_pre_election: true
I20260808 22:54:55.084005 3362 raft_consensus.cc:2454] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 0.
I20260808 22:54:55.084071 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31c68ad70943d38b2c7b691df9bdba" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" is_pre_election: true
I20260808 22:54:55.084192 3509 raft_consensus.cc:2454] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 0.
I20260808 22:54:55.084211 3147 leader_election.cc:302] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c219788fb9ed452d8228824d02688db4, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:55.084398 3596 raft_consensus.cc:2781] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:55.084466 3596 raft_consensus.cc:482] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:55.084501 3596 raft_consensus.cc:3037] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.085362 3596 raft_consensus.cc:504] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.085546 3596 leader_election.cc:288] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 election: Requested vote from peers c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:55.085708 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31c68ad70943d38b2c7b691df9bdba" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c219788fb9ed452d8228824d02688db4"
I20260808 22:54:55.085820 3362 raft_consensus.cc:3037] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.085803 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31c68ad70943d38b2c7b691df9bdba" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
I20260808 22:54:55.085872 3509 raft_consensus.cc:3037] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.086691 3362 raft_consensus.cc:2454] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 1.
I20260808 22:54:55.086691 3509 raft_consensus.cc:2454] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 1.
I20260808 22:54:55.086898 3147 leader_election.cc:302] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c219788fb9ed452d8228824d02688db4, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:55.087009 3596 raft_consensus.cc:2781] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:55.087218 3596 raft_consensus.cc:686] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 1 LEADER]: Becoming Leader. State: Replica: f218d36650f64ff5a2cd46ee04f17476, State: Running, Role: LEADER
I20260808 22:54:55.087302 3596 consensus_queue.cc:237] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.088088 3050 catalog_manager.cc:5495] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 reported cstate change: term changed from 0 to 1, leader changed from <none> to f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:54:55 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:55.129215 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.121693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.1.212.62:35351)
0808 22:54:55.121864 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.121867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.121877 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:54:55.121901 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.121902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.121919 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.122047 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:55.122538 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.123091 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.123818 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.124028 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.124118 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.126341 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:55.126343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:55.126343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:55.126351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:55.127758 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.128194 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.128195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.128196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.128276 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.128475 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.128476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.128477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.128671 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:55.128755 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.128953 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.129073 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260808 22:54:55.129271 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.121563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35351 (local address 127.0.0.1:34204)
0808 22:54:55.121742 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:55.121758 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:55.121839 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:55.122056 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:55.122058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:55.122386 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:55.122437 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.122439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.123182 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.123183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.123717 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.123719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.124119 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.124120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.124172 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.124754 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:55.124759 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:55.126244 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:55.127876 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:55.127885 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:55.127888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:55.128101 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:55.128349 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:55.128350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:55.128350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:55.128413 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:55.128746 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:55.128754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:55.128841 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 22:54:55.128929 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 22:54:55.129134 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":94,"thread_start_us":95,"threads_started":1}
I20260808 22:54:55.131554 3050 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:34204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:54:55.131773 3050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:55.133391 3189 tablet_service.cc:1467] Processing CreateTablet for tablet 18e31f39f64240ce935d96b99f6fa53f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2711633b257c43a5a8eb2379399e3246]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:55.133428 3342 tablet_service.cc:1467] Processing CreateTablet for tablet 18e31f39f64240ce935d96b99f6fa53f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2711633b257c43a5a8eb2379399e3246]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:55.133594 3189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e31f39f64240ce935d96b99f6fa53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.133595 3342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e31f39f64240ce935d96b99f6fa53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.135270 3589 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: Bootstrap starting.
I20260808 22:54:55.135630 3589 tablet_bootstrap.cc:654] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.136302 3589 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: No bootstrap required, opened a new log
I20260808 22:54:55.136379 3589 ts_tablet_manager.cc:1397] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:55.136564 3589 raft_consensus.cc:348] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.136687 3589 raft_consensus.cc:374] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.136718 3589 raft_consensus.cc:729] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c219788fb9ed452d8228824d02688db4, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.136795 3589 consensus_queue.cc:260] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.136916 3589 ts_tablet_manager.cc:1428] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:54:55.136955 3598 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.137059 3598 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.137773 3590 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: Bootstrap starting.
I20260808 22:54:55.137897 3491 tablet_service.cc:1467] Processing CreateTablet for tablet 18e31f39f64240ce935d96b99f6fa53f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2711633b257c43a5a8eb2379399e3246]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:55.138062 3491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e31f39f64240ce935d96b99f6fa53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:55.138253 3590 tablet_bootstrap.cc:654] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.139096 3590 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: No bootstrap required, opened a new log
I20260808 22:54:55.139149 3590 ts_tablet_manager.cc:1397] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:55.139305 3590 raft_consensus.cc:348] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.139411 3590 raft_consensus.cc:374] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.139437 3590 raft_consensus.cc:729] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f218d36650f64ff5a2cd46ee04f17476, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.139493 3590 consensus_queue.cc:260] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.139506 3587 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: Bootstrap starting.
I20260808 22:54:55.139583 3590 ts_tablet_manager.cc:1428] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:54:55.139712 3596 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.139776 3596 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.140020 3587 tablet_bootstrap.cc:654] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:55.140710 3587 tablet_bootstrap.cc:492] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: No bootstrap required, opened a new log
I20260808 22:54:55.140754 3587 ts_tablet_manager.cc:1397] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:54:55.140934 3587 raft_consensus.cc:348] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.140993 3587 raft_consensus.cc:374] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:55.141007 3587 raft_consensus.cc:729] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd06e8ee0747ef92d9e9cce7377545, State: Initialized, Role: FOLLOWER
I20260808 22:54:55.141062 3587 consensus_queue.cc:260] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.141156 3587 ts_tablet_manager.cc:1428] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:55.141227 3595 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.141319 3595 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.580050 3596 raft_consensus.cc:482] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:55.580173 3596 raft_consensus.cc:504] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.580374 3596 leader_election.cc:288] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:55.580611 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e31f39f64240ce935d96b99f6fa53f" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c219788fb9ed452d8228824d02688db4" is_pre_election: true
I20260808 22:54:55.580649 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e31f39f64240ce935d96b99f6fa53f" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" is_pre_election: true
I20260808 22:54:55.580751 3509 raft_consensus.cc:2454] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 0.
I20260808 22:54:55.580755 3362 raft_consensus.cc:2454] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 0.
I20260808 22:54:55.580965 3147 leader_election.cc:302] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c219788fb9ed452d8228824d02688db4, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:55.581094 3596 raft_consensus.cc:2781] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:55.581153 3596 raft_consensus.cc:482] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:55.581188 3596 raft_consensus.cc:3037] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.582047 3596 raft_consensus.cc:504] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.582234 3596 leader_election.cc:288] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 election: Requested vote from peers c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727), 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357)
I20260808 22:54:55.582427 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e31f39f64240ce935d96b99f6fa53f" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c219788fb9ed452d8228824d02688db4"
I20260808 22:54:55.582432 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e31f39f64240ce935d96b99f6fa53f" candidate_uuid: "f218d36650f64ff5a2cd46ee04f17476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd06e8ee0747ef92d9e9cce7377545"
I20260808 22:54:55.582527 3362 raft_consensus.cc:3037] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.582527 3509 raft_consensus.cc:3037] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:55.583384 3509 raft_consensus.cc:2454] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 1.
I20260808 22:54:55.583401 3362 raft_consensus.cc:2454] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f218d36650f64ff5a2cd46ee04f17476 in term 1.
I20260808 22:54:55.583618 3145 leader_election.cc:302] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ebd06e8ee0747ef92d9e9cce7377545, f218d36650f64ff5a2cd46ee04f17476; no voters:
I20260808 22:54:55.583756 3596 raft_consensus.cc:2781] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:55.583933 3596 raft_consensus.cc:686] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [term 1 LEADER]: Becoming Leader. State: Replica: f218d36650f64ff5a2cd46ee04f17476, State: Running, Role: LEADER
I20260808 22:54:55.584004 3596 consensus_queue.cc:237] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } }
I20260808 22:54:55.584528 3619 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: TxnStatusTablet state changed. Reason: New leader f218d36650f64ff5a2cd46ee04f17476. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.584671 3619 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:55.584939 3632 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:54:55.585129 3632 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:54:55.585072 3050 catalog_manager.cc:5495] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 reported cstate change: term changed from 0 to 1, leader changed from <none> to f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } health_report { overall_health: UNKNOWN } } }
I20260808 22:54:55.626658 3619 consensus_queue.cc:1035] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:55.629447 3596 consensus_queue.cc:1035] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:55.689723 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.686765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.1.212.1:39201)
0808 22:54:55.686907 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.686909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.686984 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:54:55.687086 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.687087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.687110 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.687203 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:55.687349 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.687882 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.688700 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.689031 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.689110 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.689314 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:55.689438 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:55.689519 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.689584 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.689609 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260808 22:54:55.689709 3640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.686722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39201 (local address 127.0.0.1:50342)
0808 22:54:55.686894 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:55.686906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:55.687000 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:55.687213 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:55.687214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:55.687215 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:55.687276 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.687277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.687983 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.687984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.688605 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.688606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.689123 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.689124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.689187 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.689212 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:55.689523 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:55.689534 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 22:54:55.689580 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 22:54:55.689633 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260808 22:54:55.691277 3509 raft_consensus.cc:1261] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Refusing update from remote peer f218d36650f64ff5a2cd46ee04f17476: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:55.691324 3362 raft_consensus.cc:1261] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Refusing update from remote peer f218d36650f64ff5a2cd46ee04f17476: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:55.691525 3596 consensus_queue.cc:1035] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:55.691774 3596 consensus_queue.cc:1035] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:55.692181 3598 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: TxnStatusTablet state changed. Reason: New leader f218d36650f64ff5a2cd46ee04f17476. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.692278 3598 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.692288 3595 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: TxnStatusTablet state changed. Reason: New leader f218d36650f64ff5a2cd46ee04f17476. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.692373 3595 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.693339 3619 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.693432 3619 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:55.693599 3598 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.693686 3598 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P c219788fb9ed452d8228824d02688db4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.693998 3596 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.694084 3596 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P f218d36650f64ff5a2cd46ee04f17476: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:55.695537 3595 tablet_replica.cc:439] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f218d36650f64ff5a2cd46ee04f17476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f218d36650f64ff5a2cd46ee04f17476" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39201 } } peers { permanent_uuid: "c219788fb9ed452d8228824d02688db4" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38727 } } peers { permanent_uuid: "8ebd06e8ee0747ef92d9e9cce7377545" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 44357 } } }
I20260808 22:54:55.695662 3595 tablet_replica.cc:442] T 18e31f39f64240ce935d96b99f6fa53f P 8ebd06e8ee0747ef92d9e9cce7377545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:55.699021 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.696543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.1.212.1:39201)
0808 22:54:55.696646 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.696648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.696655 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:54:55.696677 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.696678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.696687 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.696764 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:55.696947 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.697438 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.698219 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.698484 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.698565 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.698736 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:55.698774 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:55.698846 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.698904 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.698928 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 22:54:55.704854 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.701844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:34985 (local address 127.1.212.1:39201)
0808 22:54:55.701888 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.701890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.702084 (+ 194us) server_negotiation.cc:407] Connection header received
0808 22:54:55.702163 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.702163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.702172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.702244 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:55.702515 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.702988 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.703928 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.704180 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.704259 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.704476 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:55.704520 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:55.704615 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.704678 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.704719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":22}
I20260808 22:54:55.705019 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.701807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39201 (local address 127.1.212.1:34985)
0808 22:54:55.701985 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:55.702001 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:55.702093 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:55.702247 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:55.702248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:55.702250 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:55.702329 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.702330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.703085 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.703086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.703829 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:55.703832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.704291 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.704292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:55.704360 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.704379 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:55.704620 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:55.704621 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:54:55.704673 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:54:55.704739 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260808 22:54:55.722988 3050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:55.723134 3050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:55.723875 3050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9205513b70d4c8196c9b8c8425f58e9: Sending DeleteTablet for 3 replicas of tablet 7a31c68ad70943d38b2c7b691df9bdba
I20260808 22:54:55.724220 3342 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:51386
I20260808 22:54:55.724218 3189 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 22:54:55.724440 3654 tablet_replica.cc:330] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: stopping tablet replica
I20260808 22:54:55.724522 3654 raft_consensus.cc:2229] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:55.724574 3654 raft_consensus.cc:2258] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:55.724584 3491 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31c68ad70943d38b2c7b691df9bdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:56394
I20260808 22:54:55.724700 3655 tablet_replica.cc:330] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: stopping tablet replica
I20260808 22:54:55.724767 3655 raft_consensus.cc:2229] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:54:55.724828 3656 tablet_replica.cc:330] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: stopping tablet replica
I20260808 22:54:55.724908 3655 raft_consensus.cc:2258] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:55.725128 3656 raft_consensus.cc:2229] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:55.725160 3655 ts_tablet_manager.cc:1905] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.725268 3656 raft_consensus.cc:2258] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:55.725610 3656 ts_tablet_manager.cc:1905] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.725690 3654 ts_tablet_manager.cc:1905] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:55.727142 3655 ts_tablet_manager.cc:1918] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:55.727218 3655 log.cc:1196] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/7a31c68ad70943d38b2c7b691df9bdba
I20260808 22:54:55.727471 3655 ts_tablet_manager.cc:1939] T 7a31c68ad70943d38b2c7b691df9bdba P f218d36650f64ff5a2cd46ee04f17476: Deleting consensus metadata
I20260808 22:54:55.727587 3654 ts_tablet_manager.cc:1918] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:55.727664 3654 log.cc:1196] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/7a31c68ad70943d38b2c7b691df9bdba
I20260808 22:54:55.727636 3656 ts_tablet_manager.cc:1918] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:55.727837 3656 log.cc:1196] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/7a31c68ad70943d38b2c7b691df9bdba
I20260808 22:54:55.727944 3654 ts_tablet_manager.cc:1939] T 7a31c68ad70943d38b2c7b691df9bdba P c219788fb9ed452d8228824d02688db4: Deleting consensus metadata
I20260808 22:54:55.727885 3036 catalog_manager.cc:4841] TS f218d36650f64ff5a2cd46ee04f17476 (127.1.212.1:39201): tablet 7a31c68ad70943d38b2c7b691df9bdba (table test-table [id=db9c541bb1e5467dae3ebae800d6901e]) successfully deleted
I20260808 22:54:55.728340 3037 catalog_manager.cc:4841] TS c219788fb9ed452d8228824d02688db4 (127.1.212.2:38727): tablet 7a31c68ad70943d38b2c7b691df9bdba (table test-table [id=db9c541bb1e5467dae3ebae800d6901e]) successfully deleted
I20260808 22:54:55.728190 3656 ts_tablet_manager.cc:1939] T 7a31c68ad70943d38b2c7b691df9bdba P 8ebd06e8ee0747ef92d9e9cce7377545: Deleting consensus metadata
I20260808 22:54:55.728869 3035 catalog_manager.cc:4841] TS 8ebd06e8ee0747ef92d9e9cce7377545 (127.1.212.3:44357): tablet 7a31c68ad70943d38b2c7b691df9bdba (table test-table [id=db9c541bb1e5467dae3ebae800d6901e]) successfully deleted
Aug 08 22:54:55 dist-test-slave-1x4g krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:54:55.737159 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.728476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.1.212.1:39201)
0808 22:54:55.728520 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.728522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.728846 (+ 324us) server_negotiation.cc:407] Connection header received
0808 22:54:55.728989 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.728990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.729002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.729101 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:55.729792 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.730282 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.731124 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.731334 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.731411 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.733940 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:55.733951 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:55.733952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:55.733960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:55.735775 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.736186 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.736189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.736190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.736260 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.736480 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.736481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.736482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.736668 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:55.736874 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.736938 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.737034 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 22:54:55.737527 3260 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50368
I20260808 22:54:55.745412 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:55.738502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.1.212.62:35351)
0808 22:54:55.738646 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:55.738648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:55.738791 (+ 143us) server_negotiation.cc:407] Connection header received
0808 22:54:55.738867 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:55.738868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:55.738881 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:55.738933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:55.739458 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.739990 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.740845 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:55.741065 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:55.741146 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:55.742396 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:55.742398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:55.742398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:55.742406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:55.744019 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.744404 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.744405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.744406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.744481 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:55.744819 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:55.744820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:55.744820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:55.744960 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:55.745081 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:55.745190 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 22:54:55.745287 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
W20260808 22:54:55.745757 3050 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34212
I20260808 22:54:55.746619 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3127
I20260808 22:54:55.755669 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3275
I20260808 22:54:55.764181 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3422
I20260808 22:54:55.771992 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3018
2026-08-08T22:54:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3146 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:54:55 dist-test-slave-1x4g krb5kdc[3664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:54:55 dist-test-slave-1x4g krb5kdc[3664](info): set up 2 sockets
Aug 08 22:54:55 dist-test-slave-1x4g krb5kdc[3664](info): commencing operation
krb5kdc: starting...
W20260808 22:54:56.860518 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:54:56 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:54:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:54:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:54:56.996215 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33955
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:57.086323 3679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:57.086582 3679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:57.086608 3679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:57.087988 3679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:54:57.088058 3679 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:54:57.088073 3679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:57.088088 3679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:57.088100 3679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:54:57.088114 3679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:54:57.089936 3679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33955
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:57.090278 3679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:57.090574 3679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:57.096364 3684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.096352 3687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.096709 3685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:57.096853 3679 server_base.cc:1034] running on GCE node
I20260808 22:54:57.097194 3679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:57.097568 3679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:57.098732 3679 hybrid_clock.cc:648] HybridClock initialized: now 1786229697098705 us; error 43 us; skew 500 ppm
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:57.101946 3679 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:54:57.102507 3679 webserver.cc:460] Webserver started at http://127.1.212.62:44135/ using document root <none> and password file <none>
I20260808 22:54:57.102759 3679 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:57.102808 3679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:57.102957 3679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:57.104166 3679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "b7d80c67729b43bea33559cf4566b058"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "c8bf8634943ea2ee026c75a6e9523b13"
server_key_iv: "1e865dcb70aa005a2e1b736ae9fabc1d"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.104589 3679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "b7d80c67729b43bea33559cf4566b058"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "c8bf8634943ea2ee026c75a6e9523b13"
server_key_iv: "1e865dcb70aa005a2e1b736ae9fabc1d"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.106223 3679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:57.107182 3694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.107415 3679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:54:57.107542 3679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "b7d80c67729b43bea33559cf4566b058"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "c8bf8634943ea2ee026c75a6e9523b13"
server_key_iv: "1e865dcb70aa005a2e1b736ae9fabc1d"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.107635 3679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:57.124920 3679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:57.127907 3679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:57.128125 3679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:57.134065 3679 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33955
I20260808 22:54:57.134112 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33955 every 8 connection(s)
I20260808 22:54:57.134505 3679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:54:57.135213 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.137953 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Bootstrap starting.
I20260808 22:54:57.138674 3763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.138998 3763 log.cc:824] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:57.139868 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: No bootstrap required, opened a new log
I20260808 22:54:57.142374 3763 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } }
I20260808 22:54:57.142581 3763 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.142622 3763 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7d80c67729b43bea33559cf4566b058, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.142822 3763 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } }
I20260808 22:54:57.142920 3763 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:54:57.142969 3763 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:54:57.143029 3763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.143363 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3679
I20260808 22:54:57.143468 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:54:57.143630 1872 external_mini_cluster.cc:1428] Setting key e295ac1ebe1488c428465f8cc3781139
I20260808 22:54:57.143962 3763 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } }
I20260808 22:54:57.144104 3763 leader_election.cc:302] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7d80c67729b43bea33559cf4566b058; no voters:
I20260808 22:54:57.144336 3763 leader_election.cc:288] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:54:57.144379 3767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:57.144541 3767 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [term 1 LEADER]: Becoming Leader. State: Replica: b7d80c67729b43bea33559cf4566b058, State: Running, Role: LEADER
I20260808 22:54:57.144639 3763 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:54:57.144654 3767 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } }
I20260808 22:54:57.145092 3769 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7d80c67729b43bea33559cf4566b058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } } }
I20260808 22:54:57.145202 3769 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:57.145100 3770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d80c67729b43bea33559cf4566b058. Latest consensus state: current_term: 1 leader_uuid: "b7d80c67729b43bea33559cf4566b058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d80c67729b43bea33559cf4566b058" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33955 } } }
I20260808 22:54:57.146077 3770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058 [sys.catalog]: This master's current role is: LEADER
I20260808 22:54:57.146139 3776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:54:57.146703 3776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:54:57.148535 3776 catalog_manager.cc:1347] Generated new cluster ID: 7aec57414967478f8ab5c7fd29535bed
I20260808 22:54:57.148602 3776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.157186 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.144203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.1.212.62:33955)
0808 22:54:57.144533 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.144543 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.144648 (+ 105us) server_negotiation.cc:407] Connection header received
0808 22:54:57.144942 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.144950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.145025 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.145325 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.146182 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.147577 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.148679 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.149359 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.149474 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.152924 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.152930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.152931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.152946 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.155189 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.155688 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.155692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.155693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.155831 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.156073 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.156074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.156074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.156432 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.156583 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.156748 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.156925 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260808 22:54:57.180516 3776 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:54:57.181208 3776 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:54:57.186563 3776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Generated new TSK 0
I20260808 22:54:57.186807 3776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:54:57.225701 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33955
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:57.316694 3791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:57.317009 3791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:57.317040 3791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:57.318389 3791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:57.318449 3791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:57.318471 3791 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:54:57.318483 3791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:57.318502 3791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:54:57.320178 3791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:57.320469 3791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:57.320749 3791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:57.326481 3799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.326463 3796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.326504 3797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:57.326781 3791 server_base.cc:1034] running on GCE node
I20260808 22:54:57.327044 3791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:57.327332 3791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:57.328502 3791 hybrid_clock.cc:648] HybridClock initialized: now 1786229697328495 us; error 43 us; skew 500 ppm
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:57.331739 3791 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:54:57.332348 3791 webserver.cc:460] Webserver started at http://127.1.212.1:34291/ using document root <none> and password file <none>
I20260808 22:54:57.332636 3791 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:57.332685 3791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:57.332871 3791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:57.334043 3791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "2a4a08d4253a48d4af892fabaa3f3cf8"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "02f3a62c1c3a6203159b0fb5411ec54e"
server_key_iv: "3fd1318e3837e0b3ef7ed0c39f73ae56"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.334446 3791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "2a4a08d4253a48d4af892fabaa3f3cf8"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "02f3a62c1c3a6203159b0fb5411ec54e"
server_key_iv: "3fd1318e3837e0b3ef7ed0c39f73ae56"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.335954 3791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:54:57.336752 3806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.336974 3791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:57.337076 3791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "2a4a08d4253a48d4af892fabaa3f3cf8"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "02f3a62c1c3a6203159b0fb5411ec54e"
server_key_iv: "3fd1318e3837e0b3ef7ed0c39f73ae56"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.337177 3791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:57.354419 3791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:57.357129 3791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:57.357318 3791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:57.357913 3791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:57.357980 3791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.358026 3791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:57.358073 3791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.365538 3791 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:38355
I20260808 22:54:57.365631 3927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:38355 every 8 connection(s)
I20260808 22:54:57.366010 3791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.369597 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.358521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.1:50645)
0808 22:54:57.358728 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.358766 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.359324 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.359692 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.359699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.360155 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.360503 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.360512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.361655 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.361656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.363278 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.363282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.363743 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.363745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.363927 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.364677 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.364690 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.366663 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.368150 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.368157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.368159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.368400 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.368595 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.368596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.368596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.368698 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.369027 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.369033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.369177 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.369274 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.369415 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260808 22:54:57.369751 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.358538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:50645 (local address 127.1.212.62:33955)
0808 22:54:57.358605 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.358608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.358854 (+ 246us) server_negotiation.cc:407] Connection header received
0808 22:54:57.359447 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.359448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.359462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.359546 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.360631 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.361521 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.363388 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.363639 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.363701 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.366773 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.366775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.366775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.366787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.368064 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.368471 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.368472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.368472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.368528 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.368755 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.368755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.368755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.368948 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.369023 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.369419 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.369573 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:54:57.370362 3928 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33955
I20260808 22:54:57.370460 3928 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:57.370669 3928 heartbeater.cc:511] Master 127.1.212.62:33955 requested a full tablet report, sending...
I20260808 22:54:57.371340 3714 ts_manager.cc:194] Registered new tserver with Master: 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355)
I20260808 22:54:57.372356 3714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:50645
I20260808 22:54:57.372975 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3791
I20260808 22:54:57.373097 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:54:57.373304 1872 external_mini_cluster.cc:1428] Setting key 28d98c06361048293fb1259f6b34ef64
I20260808 22:54:57.378544 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.371612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51847 (local address 127.1.212.62:33955)
0808 22:54:57.371662 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.371664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.371804 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:54:57.371880 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.371881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.371890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.371958 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.372406 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.373399 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.374347 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.374652 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.374734 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.375740 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.375742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.375742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.375750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.377298 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.377688 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.377689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.377690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.377750 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.377966 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.377967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.377968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.378096 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.378177 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.378343 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.378440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:54:57.378546 3934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.371533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.1:51847)
0808 22:54:57.371726 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.371739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.371819 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.371968 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.371969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.372197 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.372266 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.372268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.373524 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.373526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.374256 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.374259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.374738 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.374739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.374801 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.375316 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.375320 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.375660 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.377409 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.377411 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.377411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.377609 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.377827 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.377827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.377828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.377895 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.378168 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.378169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.378223 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.378315 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.378440 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:54:57.405007 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33955
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:57.496757 3937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:57.497072 3937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:57.497104 3937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:57.498332 3937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:57.498375 3937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:57.498397 3937 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:54:57.498409 3937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:57.498426 3937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:54:57.499938 3937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:57.500161 3937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:57.500360 3937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:57.506184 3943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.506295 3942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.506182 3945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:57.506441 3937 server_base.cc:1034] running on GCE node
I20260808 22:54:57.506748 3937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:57.507025 3937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:57.508236 3937 hybrid_clock.cc:648] HybridClock initialized: now 1786229697508215 us; error 44 us; skew 500 ppm
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:57.511138 3937 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:54:57.511701 3937 webserver.cc:460] Webserver started at http://127.1.212.2:34351/ using document root <none> and password file <none>
I20260808 22:54:57.511943 3937 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:57.511988 3937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:57.512084 3937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:57.513245 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "5c5359149cc7430785d1727c583b1639"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "04dd5653d26f024b2c6e72ea8e5ce05c"
server_key_iv: "37d4c13cdeb247401adce955bc8c1656"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.513567 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "5c5359149cc7430785d1727c583b1639"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "04dd5653d26f024b2c6e72ea8e5ce05c"
server_key_iv: "37d4c13cdeb247401adce955bc8c1656"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.515098 3937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:54:57.516007 3952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.516254 3937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:57.516355 3937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "5c5359149cc7430785d1727c583b1639"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "04dd5653d26f024b2c6e72ea8e5ce05c"
server_key_iv: "37d4c13cdeb247401adce955bc8c1656"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.516418 3937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:57.546195 3937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:57.548702 3937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:57.548851 3937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:57.549381 3937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:57.549427 3937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.549453 3937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:57.549494 3937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.557163 3937 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:35533
I20260808 22:54:57.557256 4073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:35533 every 8 connection(s)
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.557570 3937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:54:57.561216 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.550052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:48031 (local address 127.1.212.62:33955)
0808 22:54:57.550124 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.550126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.550378 (+ 252us) server_negotiation.cc:407] Connection header received
0808 22:54:57.550997 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.550999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.551013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.551100 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.552112 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.552979 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.554738 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.555023 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.555086 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.558168 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.558172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.558172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.558185 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.559681 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.560168 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.560170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.560170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.560232 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.560501 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.560501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.560502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.560628 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.560682 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.561058 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.561139 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:54:57.561339 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.550036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.2:48031)
0808 22:54:57.550246 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.550285 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.550870 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.551228 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.551233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.551671 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.551995 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.552003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.553089 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.553090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.554622 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.554627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.555128 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.555129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.555311 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.555975 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.555987 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.558066 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.559775 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.559783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.559785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.560045 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.560310 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.560310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.560311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.560436 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.560686 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.560691 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.560895 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.560990 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.561137 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":105,"threads_started":1}
I20260808 22:54:57.561930 4074 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33955
I20260808 22:54:57.562032 4074 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:57.562234 4074 heartbeater.cc:511] Master 127.1.212.62:33955 requested a full tablet report, sending...
I20260808 22:54:57.562630 3714 ts_manager.cc:194] Registered new tserver with Master: 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533)
I20260808 22:54:57.562644 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 3937
I20260808 22:54:57.562734 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:54:57.562928 1872 external_mini_cluster.cc:1428] Setting key 2ef77c79f8452861064458c0a476ca76
I20260808 22:54:57.563153 3714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:48031
I20260808 22:54:57.569965 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.563129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.2:59957)
0808 22:54:57.563287 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.563298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.563357 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.563527 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.563528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.563793 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.563858 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.563860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.564531 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.564532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.565137 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.565140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.565610 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.565611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.565680 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.566200 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.566205 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.566540 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.568495 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.568496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.568496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.568711 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.568973 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.568974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.568974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.569037 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.569532 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.569533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.569585 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.569718 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.569878 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260808 22:54:57.570031 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.563257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:59957 (local address 127.1.212.62:33955)
0808 22:54:57.563320 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.563322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.563337 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:54:57.563437 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.563438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.563450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.563508 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.563950 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.564428 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.565222 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.565510 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.565588 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.566636 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.566637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.566637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.566646 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.568380 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.568820 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.568821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.568821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.568889 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.569102 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.569104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.569104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.569335 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.569425 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.569778 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.569877 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:54:57.592278 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33955
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:54:57.684864 4083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:54:57.685145 4083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:54:57.685195 4083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:54:57.686472 4083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:54:57.686544 4083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:54:57.686581 4083 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:54:57.686650 4083 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:54:57.686694 4083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:54:57.688336 4083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:54:57.688755 4083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:54:57.689086 4083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:54:57.695124 4091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:54:57.695225 4089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:57.695147 4083 server_base.cc:1034] running on GCE node
W20260808 22:54:57.695127 4088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:54:57.695643 4083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:54:57.695936 4083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:54:57.697110 4083 hybrid_clock.cc:648] HybridClock initialized: now 1786229697697100 us; error 43 us; skew 500 ppm
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:54:57.699971 4083 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:54:57.700479 4083 webserver.cc:460] Webserver started at http://127.1.212.3:39621/ using document root <none> and password file <none>
I20260808 22:54:57.700736 4083 fs_manager.cc:359] Metadata directory not provided
I20260808 22:54:57.700850 4083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:54:57.701023 4083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:54:57.702175 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "2cb3716c9dce4f00b783cfde2da4127b"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "f3251f99aec41a31b1e4a78db6bf75d8"
server_key_iv: "6b973c53dc3436f1e8d3f468ef79d2c4"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.702637 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "2cb3716c9dce4f00b783cfde2da4127b"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "f3251f99aec41a31b1e4a78db6bf75d8"
server_key_iv: "6b973c53dc3436f1e8d3f468ef79d2c4"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.704277 4083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:54:57.705235 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.705458 4083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:54:57.705569 4083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "2cb3716c9dce4f00b783cfde2da4127b"
format_stamp: "Formatted at 2026-08-08 22:54:57 on dist-test-slave-1x4g"
server_key: "f3251f99aec41a31b1e4a78db6bf75d8"
server_key_iv: "6b973c53dc3436f1e8d3f468ef79d2c4"
server_key_version: "encryptionkey@0"
I20260808 22:54:57.705648 4083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:54:57.715341 4083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:54:57.718029 4083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:54:57.718199 4083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:54:57.718879 4083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:54:57.718926 4083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.718951 4083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:54:57.718964 4083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.727257 4083 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:39577
I20260808 22:54:57.727344 4220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:39577 every 8 connection(s)
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.727723 4083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:54:57.728989 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4083
I20260808 22:54:57.729089 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:54:57.729317 1872 external_mini_cluster.cc:1428] Setting key d90f35b384ee301b9bce8da79c955ff2
I20260808 22:54:57.731616 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.719433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:40633 (local address 127.1.212.62:33955)
0808 22:54:57.719491 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.719493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.719807 (+ 314us) server_negotiation.cc:407] Connection header received
0808 22:54:57.720668 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.720670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.720685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.720821 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.721889 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.722757 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.724735 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.725038 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.725097 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.728226 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.728228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.728229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.728239 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.730029 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.730533 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.730534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.730535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.730619 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.730908 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.730909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.730910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.731043 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.731115 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.731423 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.731522 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:54:57.731838 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.719420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.3:40633)
0808 22:54:57.719641 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.719698 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.720520 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.720933 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.720939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.721404 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.721768 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.721776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.722871 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.722872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.724594 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.724601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.725128 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.725130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.725312 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.726046 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.726061 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.728115 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.730121 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.730132 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.730135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.730450 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.730702 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.730704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.730704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.730835 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.731118 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.731126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.731284 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.731399 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.731585 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260808 22:54:57.732396 4221 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33955
I20260808 22:54:57.732497 4221 heartbeater.cc:464] Registering TS with master...
I20260808 22:54:57.732725 4221 heartbeater.cc:511] Master 127.1.212.62:33955 requested a full tablet report, sending...
I20260808 22:54:57.733155 3713 ts_manager.cc:194] Registered new tserver with Master: 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:57.733568 3713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:40633
I20260808 22:54:57.740195 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.733689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.1.212.3:51703)
0808 22:54:57.733934 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.733945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.733998 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.734129 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.734130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.734308 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.734373 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.734374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.735193 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.735194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.735721 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.735722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.736119 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.736120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.736191 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.736673 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.736678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.737193 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.738971 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.738972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.738972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.739163 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.739416 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.739417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.739418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.739486 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.739892 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.739893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.739942 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.740003 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.740117 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":90,"threads_started":1}
I20260808 22:54:57.740263 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.733755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:51703 (local address 127.1.212.62:33955)
0808 22:54:57.733806 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.733808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.734008 (+ 200us) server_negotiation.cc:407] Connection header received
0808 22:54:57.734037 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.734038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.734049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.734128 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.734532 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.735050 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.735831 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.736033 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.736100 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.737277 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.737278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.737278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.737285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.738846 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.739254 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.739255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.739255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.739323 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.739576 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.739576 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.739577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.739817 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.739875 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.740045 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.740125 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:54:57.741350 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.767717 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.759939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.1.212.62:33955)
0808 22:54:57.760007 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.760009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.760021 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:54:57.760064 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.760064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.760075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.760129 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.760713 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.761333 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.762187 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.762413 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.762476 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.765029 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.765031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.765031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.765040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.766474 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.766845 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.766846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.766846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.766921 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.767194 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.767195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.767196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.767322 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.767411 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.767568 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.767646 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:54:57.769807 3713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:57.772112 3713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:57.779235 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.774859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39577 (local address 127.0.0.1:39968)
0808 22:54:57.775064 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.775107 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.775182 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.775622 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.775626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.775628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:57.775714 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.775717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.776735 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.776736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.777949 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.777953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.778862 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.778863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.778960 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.778976 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.779076 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.779136 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
I20260808 22:54:57.779243 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.775060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.1.212.3:39577)
0808 22:54:57.775373 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.775378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.775388 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:54:57.775471 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.775474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.775493 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.775598 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:57.775812 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.776625 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.778051 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.778767 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.778863 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.778933 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.779100 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.779143 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260808 22:54:57.779521 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.774867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35533 (local address 127.0.0.1:43002)
0808 22:54:57.775238 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.775249 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.775295 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.775906 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.775907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.775909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:57.775945 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.775946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.777318 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.777319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.778306 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.778308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.779281 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.779283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.779346 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.779351 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.779385 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.779424 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":30,"thread_start_us":98,"threads_started":1}
I20260808 22:54:57.779618 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.775116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.0.0.1:48860)
0808 22:54:57.775273 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.775284 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.775344 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.775791 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.775792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.775794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:57.775869 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.775871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.777326 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.777328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.778415 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.778418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.779394 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.779395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.779444 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.779448 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.779493 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.779544 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260808 22:54:57.779903 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.775368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.1.212.1:38355)
0808 22:54:57.775560 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.775565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.775576 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:54:57.775617 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.775619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.775633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.775711 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:57.775950 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.777227 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.778509 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.779239 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.779301 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.779361 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.779727 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.779785 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260808 22:54:57.780038 4151 tablet_service.cc:1467] Processing CreateTablet for tablet 69874909866f49fe85e8dc45c2438bf8 (DEFAULT_TABLE table=test-table [id=11f6bad83ac34dbc937f93f7b5164d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.780519 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69874909866f49fe85e8dc45c2438bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.780540 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.775007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.1.212.2:35533)
0808 22:54:57.775739 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.775743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.775752 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:54:57.775791 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.775793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.775807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.776021 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:57.776082 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.777201 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.778394 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.779165 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.780292 (+ 1127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.780367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.780394 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.780440 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":114,"threads_started":1}
I20260808 22:54:57.780633 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 69874909866f49fe85e8dc45c2438bf8 (DEFAULT_TABLE table=test-table [id=11f6bad83ac34dbc937f93f7b5164d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.781009 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69874909866f49fe85e8dc45c2438bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.781203 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 69874909866f49fe85e8dc45c2438bf8 (DEFAULT_TABLE table=test-table [id=11f6bad83ac34dbc937f93f7b5164d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.781550 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69874909866f49fe85e8dc45c2438bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.784188 4247 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Bootstrap starting.
I20260808 22:54:57.784963 4247 tablet_bootstrap.cc:654] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.784979 4246 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Bootstrap starting.
I20260808 22:54:57.785382 4247 log.cc:824] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:57.785400 4248 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Bootstrap starting.
I20260808 22:54:57.785962 4246 tablet_bootstrap.cc:654] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.786221 4248 tablet_bootstrap.cc:654] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.786310 4247 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: No bootstrap required, opened a new log
I20260808 22:54:57.786314 4246 log.cc:824] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:57.786386 4247 ts_tablet_manager.cc:1397] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:54:57.786594 4248 log.cc:824] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Log is configured to *not* fsync() on all Append() calls
I20260808 22:54:57.787175 4246 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: No bootstrap required, opened a new log
I20260808 22:54:57.787271 4246 ts_tablet_manager.cc:1397] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:54:57.787468 4248 tablet_bootstrap.cc:492] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: No bootstrap required, opened a new log
I20260808 22:54:57.787555 4248 ts_tablet_manager.cc:1397] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:54:57.788733 4247 raft_consensus.cc:348] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.788942 4247 raft_consensus.cc:374] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.788995 4247 raft_consensus.cc:729] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.788872 4246 raft_consensus.cc:348] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.789050 4246 raft_consensus.cc:374] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.789105 4246 raft_consensus.cc:729] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.789162 4247 consensus_queue.cc:260] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.789225 4246 consensus_queue.cc:260] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.789456 4247 ts_tablet_manager.cc:1428] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 22:54:57.789491 3928 heartbeater.cc:503] Master 127.1.212.62:33955 was elected leader, sending a full tablet report...
I20260808 22:54:57.789523 4246 ts_tablet_manager.cc:1428] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 22:54:57.789669 4221 heartbeater.cc:503] Master 127.1.212.62:33955 was elected leader, sending a full tablet report...
I20260808 22:54:57.789829 4248 raft_consensus.cc:348] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.789988 4248 raft_consensus.cc:374] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.790042 4248 raft_consensus.cc:729] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.790176 4248 consensus_queue.cc:260] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.790467 4248 ts_tablet_manager.cc:1428] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:54:57.790619 4074 heartbeater.cc:503] Master 127.1.212.62:33955 was elected leader, sending a full tablet report...
I20260808 22:54:57.791371 4252 raft_consensus.cc:482] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:57.791442 4252 raft_consensus.cc:504] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.791707 4252 leader_election.cc:288] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577), 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533)
I20260808 22:54:57.794487 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.791837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39577 (local address 127.1.212.1:51513)
0808 22:54:57.791880 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.791893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.791934 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.792123 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.792124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.792125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:57.792198 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.792199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.792888 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.792889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.793642 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.793644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.794202 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.794203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.794283 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.794287 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.794371 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.794417 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 22:54:57.794486 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.791932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51513 (local address 127.1.212.3:39577)
0808 22:54:57.792003 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.792005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.792012 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:54:57.792034 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.792034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.792045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.792118 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:57.792277 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.792761 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.793742 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.794116 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.794202 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.794214 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.794381 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.794417 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:54:57.794787 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.791921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56817 (local address 127.1.212.2:35533)
0808 22:54:57.791964 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.791967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.792209 (+ 242us) server_negotiation.cc:407] Connection header received
0808 22:54:57.792276 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.792277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.792287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.792342 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:57.792508 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.793072 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.793999 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.794371 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.794456 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.794465 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.794659 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.794716 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:54:57.794986 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69874909866f49fe85e8dc45c2438bf8" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b" is_pre_election: true
I20260808 22:54:57.795085 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.791843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35533 (local address 127.1.212.1:56817)
0808 22:54:57.792127 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.792137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.792224 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.792346 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.792347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.792348 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:57.792426 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.792428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.793162 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.793163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.793906 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.793908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.794458 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.794459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.794559 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.794564 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.794933 (+ 369us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.794977 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":120,"thread_start_us":119,"threads_started":1}
I20260808 22:54:57.795140 4172 raft_consensus.cc:2454] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:57.795395 3810 leader_election.cc:302] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 2cb3716c9dce4f00b783cfde2da4127b; no voters:
I20260808 22:54:57.795430 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69874909866f49fe85e8dc45c2438bf8" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639" is_pre_election: true
I20260808 22:54:57.795545 4252 raft_consensus.cc:2781] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:57.795595 4252 raft_consensus.cc:482] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:57.795593 4009 raft_consensus.cc:2454] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:57.795632 4252 raft_consensus.cc:3037] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.796569 4252 raft_consensus.cc:504] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.796741 4252 leader_election.cc:288] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Requested vote from peers 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577), 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533)
I20260808 22:54:57.796933 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69874909866f49fe85e8dc45c2438bf8" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639"
I20260808 22:54:57.797048 4009 raft_consensus.cc:3037] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.797820 4009 raft_consensus.cc:2454] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:57.797972 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69874909866f49fe85e8dc45c2438bf8" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b"
I20260808 22:54:57.798110 4172 raft_consensus.cc:3037] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.798468 3809 leader_election.cc:302] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:57.798641 4252 raft_consensus.cc:2781] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:57.798863 4252 raft_consensus.cc:686] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Running, Role: LEADER
I20260808 22:54:57.798983 4252 consensus_queue.cc:237] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } }
I20260808 22:54:57.799190 4172 raft_consensus.cc:2454] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:57.799938 3711 catalog_manager.cc:5495] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } health_report { overall_health: HEALTHY } } }
W20260808 22:54:57.808176 4075 tablet.cc:2369] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:54:57 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:57.840828 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.833716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.1.212.62:33955)
0808 22:54:57.833791 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.833793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.833898 (+ 105us) server_negotiation.cc:407] Connection header received
0808 22:54:57.833950 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.833951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.833961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.834038 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:57.834513 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.835000 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.835720 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.835902 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.835958 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.838204 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:57.838206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:57.838206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:57.838214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:57.839492 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.839920 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.839921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.839921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.839973 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:57.840220 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:57.840221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:57.840221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:57.840339 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:57.840421 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.840592 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.840703 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:54:57.840911 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.833637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33955 (local address 127.0.0.1:37834)
0808 22:54:57.833818 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.833834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.833901 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.834058 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.834059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.834371 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:54:57.834426 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.834429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.835091 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.835092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.835634 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.835635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.835984 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.835985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.836037 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.836561 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:54:57.836566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:54:57.838112 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:54:57.839612 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.839621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.839624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.839847 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.840071 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:54:57.840072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:54:57.840072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:54:57.840140 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:54:57.840410 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:54:57.840420 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:54:57.840469 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.840565 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.840770 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":87,"thread_start_us":106,"threads_started":1}
I20260808 22:54:57.843204 3711 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:37834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:54:57.843412 3711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:57.845134 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 82b4319a7ab341e1853a9be97a938004 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8dc263cf828846a288c52139fee5d2d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:57.845170 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 82b4319a7ab341e1853a9be97a938004 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8dc263cf828846a288c52139fee5d2d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:57.845155 4151 tablet_service.cc:1467] Processing CreateTablet for tablet 82b4319a7ab341e1853a9be97a938004 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8dc263cf828846a288c52139fee5d2d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:54:57.845337 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b4319a7ab341e1853a9be97a938004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.845336 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b4319a7ab341e1853a9be97a938004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.845363 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b4319a7ab341e1853a9be97a938004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.846925 4246 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: Bootstrap starting.
I20260808 22:54:57.847003 4248 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: Bootstrap starting.
I20260808 22:54:57.847499 4246 tablet_bootstrap.cc:654] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.847620 4248 tablet_bootstrap.cc:654] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.847834 4247 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: Bootstrap starting.
I20260808 22:54:57.848222 4246 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: No bootstrap required, opened a new log
I20260808 22:54:57.848276 4246 ts_tablet_manager.cc:1397] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:57.848322 4247 tablet_bootstrap.cc:654] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.848387 4248 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: No bootstrap required, opened a new log
I20260808 22:54:57.848424 4248 ts_tablet_manager.cc:1397] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:57.848438 4246 raft_consensus.cc:348] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.848534 4246 raft_consensus.cc:374] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.848554 4246 raft_consensus.cc:729] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.848569 4248 raft_consensus.cc:348] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.848603 4246 consensus_queue.cc:260] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.848649 4248 raft_consensus.cc:374] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.848673 4248 raft_consensus.cc:729] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.848702 4246 ts_tablet_manager.cc:1428] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:54:57.848716 4248 consensus_queue.cc:260] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.848858 4248 ts_tablet_manager.cc:1428] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:54:57.848924 4253 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.848927 4254 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.849020 4254 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.849018 4253 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.849725 4247 tablet_bootstrap.cc:492] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: No bootstrap required, opened a new log
I20260808 22:54:57.849771 4247 ts_tablet_manager.cc:1397] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260808 22:54:57.849951 4247 raft_consensus.cc:348] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.850020 4247 raft_consensus.cc:374] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.850072 4247 raft_consensus.cc:729] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.850128 4247 consensus_queue.cc:260] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.850224 4247 ts_tablet_manager.cc:1428] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.850356 4252 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.850432 4252 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 22:54:57.866978 3929 tablet.cc:2369] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:54:57.896428 4252 raft_consensus.cc:482] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:57.896536 4252 raft_consensus.cc:504] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.896768 4252 leader_election.cc:288] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:57.897025 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b4319a7ab341e1853a9be97a938004" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b" is_pre_election: true
I20260808 22:54:57.897123 4172 raft_consensus.cc:2454] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:57.897047 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b4319a7ab341e1853a9be97a938004" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639" is_pre_election: true
I20260808 22:54:57.897173 4009 raft_consensus.cc:2454] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:57.897310 3810 leader_election.cc:302] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 2cb3716c9dce4f00b783cfde2da4127b; no voters:
I20260808 22:54:57.897431 4252 raft_consensus.cc:2781] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:57.897488 4252 raft_consensus.cc:482] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:57.897509 4252 raft_consensus.cc:3037] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.898196 4252 raft_consensus.cc:504] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.898346 4252 leader_election.cc:288] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Requested vote from peers 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:57.898558 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b4319a7ab341e1853a9be97a938004" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639"
I20260808 22:54:57.898571 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b4319a7ab341e1853a9be97a938004" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b"
I20260808 22:54:57.898663 4009 raft_consensus.cc:3037] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.898661 4172 raft_consensus.cc:3037] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:57.899313 4172 raft_consensus.cc:2454] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:57.899416 4009 raft_consensus.cc:2454] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:57.899518 3810 leader_election.cc:302] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 2cb3716c9dce4f00b783cfde2da4127b; no voters:
I20260808 22:54:57.899655 4252 raft_consensus.cc:2781] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:57.899721 4252 raft_consensus.cc:686] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Running, Role: LEADER
I20260808 22:54:57.899763 4252 consensus_queue.cc:237] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.900172 4257 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: TxnStatusTablet state changed. Reason: New leader 2a4a08d4253a48d4af892fabaa3f3cf8. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.900280 4257 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:57.900490 4269 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:54:57.900635 4269 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:54:57.900574 3714 catalog_manager.cc:5495] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260808 22:54:57.909612 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.906615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.1.212.1:38355)
0808 22:54:57.906672 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.906674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.906759 (+ 85us) server_negotiation.cc:407] Connection header received
0808 22:54:57.906873 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.906874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.906892 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.906952 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:57.907113 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.907609 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.908751 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.909002 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.909058 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.909284 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:57.909384 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:57.909446 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.909528 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.909548 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:54:57.909667 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.906527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.0.0.1:48872)
0808 22:54:57.906700 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.906714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.906785 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.906981 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.906982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.906984 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:57.907046 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.907049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.907695 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.907697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.908626 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.908630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.909087 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.909088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.909160 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.909182 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:57.909447 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:57.909458 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.909527 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.909580 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 22:54:57.911872 4172 raft_consensus.cc:1261] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:57.911873 4009 raft_consensus.cc:1261] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:57.912262 4257 consensus_queue.cc:1035] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:57.912447 4252 consensus_queue.cc:1035] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:57.915719 4254 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: TxnStatusTablet state changed. Reason: New leader 2a4a08d4253a48d4af892fabaa3f3cf8. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.915838 4254 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.914999 4253 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: TxnStatusTablet state changed. Reason: New leader 2a4a08d4253a48d4af892fabaa3f3cf8. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.916141 4253 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.917407 4257 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.917503 4257 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:57.917614 4257 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.917675 4257 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2a4a08d4253a48d4af892fabaa3f3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:54:57.917668 4254 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.917750 4254 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 5c5359149cc7430785d1727c583b1639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.917819 4272 mvcc.cc:205] Tried to move back new op lower bound from 7316396842644041728 to 7316396842598129664. Current Snapshot: MvccSnapshot[applied={T|T < 7316396842644041728}]
I20260808 22:54:57.918529 4253 tablet_replica.cc:439] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } } }
I20260808 22:54:57.918627 4253 tablet_replica.cc:442] T 82b4319a7ab341e1853a9be97a938004 P 2cb3716c9dce4f00b783cfde2da4127b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:54:57.919147 4275 mvcc.cc:205] Tried to move back new op lower bound from 7316396842644041728 to 7316396842598129664. Current Snapshot: MvccSnapshot[applied={T|T < 7316396842644041728}]
I20260808 22:54:57.925222 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.922775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.1.212.1:38355)
0808 22:54:57.922820 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.922822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.922897 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:54:57.922976 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.922977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.922985 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.923050 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:57.923207 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.923696 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.924495 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.924715 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.924820 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.924944 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:57.924983 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:57.925050 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.925114 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.925136 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:54:57.931686 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.928635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:34551 (local address 127.1.212.1:38355)
0808 22:54:57.928701 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:57.928703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:57.928938 (+ 235us) server_negotiation.cc:407] Connection header received
0808 22:54:57.928998 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:57.928999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:57.929009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:57.929058 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:57.929234 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.929686 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.930932 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.931116 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.931171 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.931402 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:57.931447 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:57.931486 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:57.931577 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:54:57.931603 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":40}
I20260808 22:54:57.931867 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:57.928570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.1.212.1:34551)
0808 22:54:57.928770 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:57.928805 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:57.928937 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:57.929084 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:57.929085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:57.929087 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:57.929169 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.929171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.929765 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.929766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.930818 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:57.930823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:57.931194 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:57.931195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:57.931267 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:57.931291 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:57.931510 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:57.931512 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:54:57.931575 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:54:57.931630 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260808 22:54:57.934731 4172 raft_consensus.cc:1261] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:57.934775 4009 raft_consensus.cc:1261] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:57.935029 4284 consensus_queue.cc:1035] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:57.935133 4252 consensus_queue.cc:1035] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:57.936712 4273 mvcc.cc:205] Tried to move back new op lower bound from 7316396842739339264 to 7316396842185256960. Current Snapshot: MvccSnapshot[applied={T|T < 7316396842739339264}]
I20260808 22:54:57.951016 3711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:57.951237 3711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:57.952157 3711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Sending DeleteTablet for 3 replicas of tablet 69874909866f49fe85e8dc45c2438bf8
I20260808 22:54:57.952495 4151 tablet_service.cc:1514] Processing DeleteTablet for tablet 69874909866f49fe85e8dc45c2438bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:39968
I20260808 22:54:57.952548 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 69874909866f49fe85e8dc45c2438bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43002
I20260808 22:54:57.952643 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 69874909866f49fe85e8dc45c2438bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:48860
I20260808 22:54:57.952812 4288 tablet_replica.cc:330] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: stopping tablet replica
I20260808 22:54:57.952940 4289 tablet_replica.cc:330] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: stopping tablet replica
I20260808 22:54:57.952944 4288 raft_consensus.cc:2229] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:57.953024 4288 raft_consensus.cc:2258] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:57.953059 4289 raft_consensus.cc:2229] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:57.953214 4289 raft_consensus.cc:2258] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:57.953311 3711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:57.953472 3711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:57.953681 4289 ts_tablet_manager.cc:1905] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:57.954787 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 (DEFAULT_TABLE table=test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.954787 4151 tablet_service.cc:1467] Processing CreateTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 (DEFAULT_TABLE table=test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.954818 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 (DEFAULT_TABLE table=test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:57.954914 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a5bfd37bc14d09bab319b5383096f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.954926 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a5bfd37bc14d09bab319b5383096f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.954958 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a5bfd37bc14d09bab319b5383096f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:57.956280 4248 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Bootstrap starting.
I20260808 22:54:57.956441 4246 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Bootstrap starting.
I20260808 22:54:57.956763 4248 tablet_bootstrap.cc:654] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.956847 4247 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Bootstrap starting.
I20260808 22:54:57.957275 4247 tablet_bootstrap.cc:654] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.957365 4288 ts_tablet_manager.cc:1905] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:57.957792 4246 tablet_bootstrap.cc:654] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:57.958499 4246 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: No bootstrap required, opened a new log
I20260808 22:54:57.958567 4246 ts_tablet_manager.cc:1397] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:54:57.958729 4246 raft_consensus.cc:348] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.958817 4246 raft_consensus.cc:374] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.958837 4246 raft_consensus.cc:729] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.958886 4246 consensus_queue.cc:260] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.958986 4246 ts_tablet_manager.cc:1428] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.959488 4289 ts_tablet_manager.cc:1918] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:57.959578 4288 ts_tablet_manager.cc:1918] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:57.959656 4288 log.cc:1196] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/69874909866f49fe85e8dc45c2438bf8
I20260808 22:54:57.959658 4289 log.cc:1196] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/69874909866f49fe85e8dc45c2438bf8
I20260808 22:54:57.959944 4288 ts_tablet_manager.cc:1939] T 69874909866f49fe85e8dc45c2438bf8 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting consensus metadata
I20260808 22:54:57.960147 4289 ts_tablet_manager.cc:1939] T 69874909866f49fe85e8dc45c2438bf8 P 5c5359149cc7430785d1727c583b1639: Deleting consensus metadata
I20260808 22:54:57.960464 3699 catalog_manager.cc:4841] TS 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577): tablet 69874909866f49fe85e8dc45c2438bf8 (table test-table [id=11f6bad83ac34dbc937f93f7b5164d89]) successfully deleted
I20260808 22:54:57.960573 3698 catalog_manager.cc:4841] TS 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533): tablet 69874909866f49fe85e8dc45c2438bf8 (table test-table [id=11f6bad83ac34dbc937f93f7b5164d89]) successfully deleted
I20260808 22:54:57.960918 4290 tablet_replica.cc:330] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: stopping tablet replica
I20260808 22:54:57.960932 4247 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: No bootstrap required, opened a new log
I20260808 22:54:57.960996 4247 ts_tablet_manager.cc:1397] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 22:54:57.961055 4248 tablet_bootstrap.cc:492] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: No bootstrap required, opened a new log
I20260808 22:54:57.961100 4248 ts_tablet_manager.cc:1397] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 22:54:57.961186 4247 raft_consensus.cc:348] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.961277 4247 raft_consensus.cc:374] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.961302 4247 raft_consensus.cc:729] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.961264 4248 raft_consensus.cc:348] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.961334 4248 raft_consensus.cc:374] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:57.961355 4248 raft_consensus.cc:729] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Initialized, Role: FOLLOWER
I20260808 22:54:57.961362 4247 consensus_queue.cc:260] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.961397 4248 consensus_queue.cc:260] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:57.961418 4290 raft_consensus.cc:2229] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:54:57.961463 4247 ts_tablet_manager.cc:1428] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:54:57.961486 4248 ts_tablet_manager.cc:1428] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:54:57.961540 4290 raft_consensus.cc:2258] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:57.962414 4290 ts_tablet_manager.cc:1905] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:57.964221 4290 ts_tablet_manager.cc:1918] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:57.964306 4290 log.cc:1196] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/69874909866f49fe85e8dc45c2438bf8
I20260808 22:54:57.964578 4290 ts_tablet_manager.cc:1939] T 69874909866f49fe85e8dc45c2438bf8 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting consensus metadata
I20260808 22:54:57.965059 3698 catalog_manager.cc:4841] TS 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355): tablet 69874909866f49fe85e8dc45c2438bf8 (table test-table [id=11f6bad83ac34dbc937f93f7b5164d89]) successfully deleted
W20260808 22:54:57.978354 4222 tablet.cc:2369] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:54:58.001922 4254 raft_consensus.cc:482] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:58.002069 4254 raft_consensus.cc:504] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:58.002494 4254 leader_election.cc:288] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:58.006335 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.002669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.1.212.2:35405)
0808 22:54:58.002726 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:58.002739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:58.002782 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:58.002965 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:58.002966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:58.002969 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:58.003051 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.003053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.004072 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.004073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.005343 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.005347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.006095 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.006095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.006179 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.006182 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:54:58.006232 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:54:58.006264 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":8}
I20260808 22:54:58.006356 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.002891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:44813 (local address 127.1.212.3:39577)
0808 22:54:58.002940 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:58.002942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:58.003071 (+ 129us) server_negotiation.cc:407] Connection header received
0808 22:54:58.003138 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:58.003139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:58.003148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:58.003194 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:58.003665 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.004318 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.005450 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.005860 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.005961 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.005974 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:58.006224 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 22:54:58.006281 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:54:58.006341 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.002755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39577 (local address 127.1.212.2:44813)
0808 22:54:58.002992 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:58.003001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:58.003082 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:58.003206 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:58.003207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:58.003208 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:58.003574 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.003577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.004441 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.004443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.005355 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.005359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.005981 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.005982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.006073 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.006078 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:54:58.006182 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 22:54:58.006238 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":62,"threads_started":1}
I20260808 22:54:58.006687 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.002724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:35405 (local address 127.1.212.1:38355)
0808 22:54:58.002826 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:58.002828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:58.002836 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:54:58.002863 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:58.002864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:58.002876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:58.002941 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:58.003158 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.003914 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.005462 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.006008 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.006090 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.006104 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:58.006549 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 22:54:58.006614 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 22:54:58.006830 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a5bfd37bc14d09bab319b5383096f0" candidate_uuid: "5c5359149cc7430785d1727c583b1639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b" is_pre_election: true
I20260808 22:54:58.006976 4172 raft_consensus.cc:2454] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5359149cc7430785d1727c583b1639 in term 0.
I20260808 22:54:58.007112 3880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a5bfd37bc14d09bab319b5383096f0" candidate_uuid: "5c5359149cc7430785d1727c583b1639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" is_pre_election: true
I20260808 22:54:58.007184 3956 leader_election.cc:302] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb3716c9dce4f00b783cfde2da4127b, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:58.007279 3880 raft_consensus.cc:2454] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5359149cc7430785d1727c583b1639 in term 0.
I20260808 22:54:58.007390 4254 raft_consensus.cc:2781] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:58.007437 4254 raft_consensus.cc:482] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:58.007460 4254 raft_consensus.cc:3037] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:58.008212 4254 raft_consensus.cc:504] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:58.008368 4254 leader_election.cc:288] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [CANDIDATE]: Term 1 election: Requested vote from peers 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:58.008548 3880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a5bfd37bc14d09bab319b5383096f0" candidate_uuid: "5c5359149cc7430785d1727c583b1639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8"
I20260808 22:54:58.008569 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a5bfd37bc14d09bab319b5383096f0" candidate_uuid: "5c5359149cc7430785d1727c583b1639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b"
I20260808 22:54:58.008644 3880 raft_consensus.cc:3037] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:58.008667 4172 raft_consensus.cc:3037] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:58.009428 4172 raft_consensus.cc:2454] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5359149cc7430785d1727c583b1639 in term 1.
I20260808 22:54:58.009604 3880 raft_consensus.cc:2454] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5359149cc7430785d1727c583b1639 in term 1.
I20260808 22:54:58.009665 3956 leader_election.cc:302] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb3716c9dce4f00b783cfde2da4127b, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:58.009802 4254 raft_consensus.cc:2781] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:58.010004 4254 raft_consensus.cc:686] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 1 LEADER]: Becoming Leader. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Running, Role: LEADER
I20260808 22:54:58.010092 4254 consensus_queue.cc:237] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:58.010922 3714 catalog_manager.cc:5495] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c5359149cc7430785d1727c583b1639 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "5c5359149cc7430785d1727c583b1639" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260808 22:54:58.020917 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.018165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.1.212.2:35533)
0808 22:54:58.018208 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:58.018210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:58.018220 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:54:58.018290 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:58.018291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:58.018311 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:58.018418 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:58.018555 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.019197 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.019963 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.020241 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.020318 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.020484 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:58.020638 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:58.020732 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:58.020752 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 22:54:58.020819 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:54:58.026005 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.023532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.1.212.2:38397)
0808 22:54:58.023608 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:58.023620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:58.023662 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:58.023798 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:58.023799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:58.023800 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:58.023888 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.023890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.024518 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.024519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.025181 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.025183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.025555 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.025556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.025624 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.025647 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:58.025837 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:58.025840 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:54:58.025887 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 22:54:58.025936 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:54:58.026000 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.023627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:38397 (local address 127.1.212.1:38355)
0808 22:54:58.023677 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:58.023679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:58.023688 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:54:58.023716 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:58.023716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:58.023726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:58.023796 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:58.023973 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.024434 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.025269 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.025467 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.025553 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.025732 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:58.025774 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:58.025835 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:58.025894 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 22:54:58.025916 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:54:58.028828 3880 raft_consensus.cc:1261] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Refusing update from remote peer 5c5359149cc7430785d1727c583b1639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:58.028829 4172 raft_consensus.cc:1261] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Refusing update from remote peer 5c5359149cc7430785d1727c583b1639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:58.029165 4297 consensus_queue.cc:1035] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:58.029307 4254 consensus_queue.cc:1035] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:58.038591 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.035988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:35889 (local address 127.1.212.2:35533)
0808 22:54:58.036077 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:58.036079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:58.036093 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:54:58.036116 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:58.036117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:58.036130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:58.036214 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:58.036326 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.036909 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.037821 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.038117 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.038200 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.038333 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:58.038370 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:58.038432 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:58.038488 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 22:54:58.038511 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:54:58.038606 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:58.035759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35533 (local address 127.1.212.1:35889)
0808 22:54:58.035811 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:58.035823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:58.035862 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:58.036218 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:58.036220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:58.036221 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:58.036273 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.036274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.037001 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.037003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.037733 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:58.037736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:58.038194 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:58.038195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:58.038242 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:58.038249 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:58.038430 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:58.038432 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:54:58.038483 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 22:54:58.038536 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 22:54:58.038992 4275 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:54:59.566735 3711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:59.566854 3711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:59.567759 3711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Sending DeleteTablet for 3 replicas of tablet 28a5bfd37bc14d09bab319b5383096f0
I20260808 22:54:59.568106 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43002
I20260808 22:54:59.568127 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:48860
I20260808 22:54:59.568153 4151 tablet_service.cc:1514] Processing DeleteTablet for tablet 28a5bfd37bc14d09bab319b5383096f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:39968
I20260808 22:54:59.571445 4326 tablet_replica.cc:330] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: stopping tablet replica
I20260808 22:54:59.571580 4326 raft_consensus.cc:2229] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:54:59.571655 4328 tablet_replica.cc:330] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: stopping tablet replica
I20260808 22:54:59.571708 4326 raft_consensus.cc:2258] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.571728 4328 raft_consensus.cc:2229] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:59.571782 4328 raft_consensus.cc:2258] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.571906 4327 tablet_replica.cc:330] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: stopping tablet replica
I20260808 22:54:59.571978 4327 raft_consensus.cc:2229] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:59.572027 4327 raft_consensus.cc:2258] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.572052 4326 ts_tablet_manager.cc:1905] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.572257 4327 ts_tablet_manager.cc:1905] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.574132 4328 ts_tablet_manager.cc:1905] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.574301 4326 ts_tablet_manager.cc:1918] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:54:59.574287 4327 ts_tablet_manager.cc:1918] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:54:59.574365 4327 log.cc:1196] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/28a5bfd37bc14d09bab319b5383096f0
I20260808 22:54:59.574422 4326 log.cc:1196] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/28a5bfd37bc14d09bab319b5383096f0
I20260808 22:54:59.574725 4327 ts_tablet_manager.cc:1939] T 28a5bfd37bc14d09bab319b5383096f0 P 2cb3716c9dce4f00b783cfde2da4127b: Deleting consensus metadata
I20260808 22:54:59.574852 4326 ts_tablet_manager.cc:1939] T 28a5bfd37bc14d09bab319b5383096f0 P 5c5359149cc7430785d1727c583b1639: Deleting consensus metadata
I20260808 22:54:59.575163 3699 catalog_manager.cc:4841] TS 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577): tablet 28a5bfd37bc14d09bab319b5383096f0 (table test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]) successfully deleted
I20260808 22:54:59.575263 3698 catalog_manager.cc:4841] TS 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533): tablet 28a5bfd37bc14d09bab319b5383096f0 (table test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]) successfully deleted
I20260808 22:54:59.575915 4328 ts_tablet_manager.cc:1918] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:54:59.575973 4328 log.cc:1196] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/28a5bfd37bc14d09bab319b5383096f0
I20260808 22:54:59.576208 4328 ts_tablet_manager.cc:1939] T 28a5bfd37bc14d09bab319b5383096f0 P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting consensus metadata
I20260808 22:54:59.576579 3698 catalog_manager.cc:4841] TS 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355): tablet 28a5bfd37bc14d09bab319b5383096f0 (table test-table [id=a5d8a7ea476c4d379dd7547e0d091fbb]) successfully deleted
Aug 08 22:54:59 dist-test-slave-1x4g krb5kdc[3664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:54:59 dist-test-slave-1x4g krb5kdc[3664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:54:59.596545 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.588471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.1.212.62:33955)
0808 22:54:59.588648 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.588650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.588662 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:54:59.588721 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.588722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.588733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.588849 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:54:59.589464 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.590024 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.590777 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.591005 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.591074 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.593684 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:54:59.593686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:54:59.593686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:54:59.593694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:54:59.595353 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:59.595742 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:59.595743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:59.595743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:59.595802 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:54:59.595979 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:54:59.595980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:54:59.595980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:54:59.596148 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:54:59.596233 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.596382 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.596458 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260808 22:54:59.598541 3711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:59.598721 3711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:59.600384 4151 tablet_service.cc:1467] Processing CreateTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc (DEFAULT_TABLE table=test-table [id=cba998ca05094e978c3c9f180608aebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.600384 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc (DEFAULT_TABLE table=test-table [id=cba998ca05094e978c3c9f180608aebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.600406 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc (DEFAULT_TABLE table=test-table [id=cba998ca05094e978c3c9f180608aebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.600569 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34ae651c379a48f5b45856fcbe42d0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.600569 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34ae651c379a48f5b45856fcbe42d0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.600569 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34ae651c379a48f5b45856fcbe42d0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.601873 4338 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Bootstrap starting.
I20260808 22:54:59.602310 4340 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Bootstrap starting.
I20260808 22:54:59.602339 4338 tablet_bootstrap.cc:654] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.602538 4341 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Bootstrap starting.
I20260808 22:54:59.602803 4340 tablet_bootstrap.cc:654] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.602910 4341 tablet_bootstrap.cc:654] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.603096 4338 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: No bootstrap required, opened a new log
I20260808 22:54:59.603206 4338 ts_tablet_manager.cc:1397] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.603346 4338 raft_consensus.cc:348] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.603472 4338 raft_consensus.cc:374] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.603530 4338 raft_consensus.cc:729] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.603560 4341 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: No bootstrap required, opened a new log
I20260808 22:54:59.603561 4340 tablet_bootstrap.cc:492] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: No bootstrap required, opened a new log
I20260808 22:54:59.603612 4340 ts_tablet_manager.cc:1397] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.603645 4341 ts_tablet_manager.cc:1397] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.603636 4338 consensus_queue.cc:260] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.603739 4340 raft_consensus.cc:348] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.603777 4341 raft_consensus.cc:348] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.603818 4340 raft_consensus.cc:374] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.603839 4341 raft_consensus.cc:374] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.603843 4340 raft_consensus.cc:729] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.603863 4341 raft_consensus.cc:729] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.603896 4340 consensus_queue.cc:260] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.603999 4341 consensus_queue.cc:260] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.604169 4340 ts_tablet_manager.cc:1428] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.604346 4341 ts_tablet_manager.cc:1428] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:54:59.604530 4338 ts_tablet_manager.cc:1428] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:54:59.685514 4252 raft_consensus.cc:482] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:59.685629 4252 raft_consensus.cc:504] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.685856 4252 leader_election.cc:288] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:59.686276 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34ae651c379a48f5b45856fcbe42d0fc" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b" is_pre_election: true
I20260808 22:54:59.686270 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34ae651c379a48f5b45856fcbe42d0fc" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639" is_pre_election: true
I20260808 22:54:59.686399 4172 raft_consensus.cc:2454] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:59.686406 4009 raft_consensus.cc:2454] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 0.
I20260808 22:54:59.686594 3809 leader_election.cc:302] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:59.686724 4252 raft_consensus.cc:2781] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:59.686789 4252 raft_consensus.cc:482] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:59.686810 4252 raft_consensus.cc:3037] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.687522 4252 raft_consensus.cc:504] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.687691 4252 leader_election.cc:288] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Requested vote from peers 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533), 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577)
I20260808 22:54:59.687888 4009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34ae651c379a48f5b45856fcbe42d0fc" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639"
I20260808 22:54:59.687888 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34ae651c379a48f5b45856fcbe42d0fc" candidate_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb3716c9dce4f00b783cfde2da4127b"
I20260808 22:54:59.687981 4172 raft_consensus.cc:3037] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.687981 4009 raft_consensus.cc:3037] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.688901 4172 raft_consensus.cc:2454] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:59.688900 4009 raft_consensus.cc:2454] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4a08d4253a48d4af892fabaa3f3cf8 in term 1.
I20260808 22:54:59.689131 3810 leader_election.cc:302] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4a08d4253a48d4af892fabaa3f3cf8, 2cb3716c9dce4f00b783cfde2da4127b; no voters:
I20260808 22:54:59.689277 4252 raft_consensus.cc:2781] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:59.689333 4252 raft_consensus.cc:686] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Running, Role: LEADER
I20260808 22:54:59.689396 4252 consensus_queue.cc:237] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.690186 3714 catalog_manager.cc:5495] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260808 22:54:59.701619 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.698984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.1.212.1:38355)
0808 22:54:59.699177 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.699186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.699192 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:54:59.699214 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.699214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.699225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.699303 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:59.699462 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.699980 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.700812 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.701047 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.701144 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.701308 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.701354 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.701419 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.701486 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.701514 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260808 22:54:59.704283 4172 raft_consensus.cc:1261] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:59.704334 4009 raft_consensus.cc:1261] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Refusing update from remote peer 2a4a08d4253a48d4af892fabaa3f3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:59.704537 4252 consensus_queue.cc:1035] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:59.704658 4307 consensus_queue.cc:1035] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:59.715557 3711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:59.715693 3711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:54:59.716387 3711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Sending DeleteTablet for 3 replicas of tablet 34ae651c379a48f5b45856fcbe42d0fc
I20260808 22:54:59.716681 4151 tablet_service.cc:1514] Processing DeleteTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:39968
I20260808 22:54:59.716681 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:48860
I20260808 22:54:59.716735 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 34ae651c379a48f5b45856fcbe42d0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:54:59 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43002
I20260808 22:54:59.716821 4328 tablet_replica.cc:330] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: stopping tablet replica
I20260808 22:54:59.716869 4326 tablet_replica.cc:330] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: stopping tablet replica
I20260808 22:54:59.716890 4328 raft_consensus.cc:2229] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:54:59.716882 4327 tablet_replica.cc:330] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: stopping tablet replica
I20260808 22:54:59.716933 4326 raft_consensus.cc:2229] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:59.716953 4327 raft_consensus.cc:2229] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:54:59.716975 4326 raft_consensus.cc:2258] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.716964 4328 raft_consensus.cc:2258] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.717005 4327 raft_consensus.cc:2258] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:54:59.717085 3711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:54:59.717252 3711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:54:59.717357 4327 ts_tablet_manager.cc:1905] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.717526 4326 ts_tablet_manager.cc:1905] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.717684 4328 ts_tablet_manager.cc:1905] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:54:59.719249 4326 ts_tablet_manager.cc:1918] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:59.719193 4327 ts_tablet_manager.cc:1918] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:59.719329 4327 log.cc:1196] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/34ae651c379a48f5b45856fcbe42d0fc
I20260808 22:54:59.719331 4326 log.cc:1196] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/34ae651c379a48f5b45856fcbe42d0fc
I20260808 22:54:59.719408 4328 ts_tablet_manager.cc:1918] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:54:59.719485 4328 log.cc:1196] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/34ae651c379a48f5b45856fcbe42d0fc
I20260808 22:54:59.719589 4326 ts_tablet_manager.cc:1939] T 34ae651c379a48f5b45856fcbe42d0fc P 5c5359149cc7430785d1727c583b1639: Deleting consensus metadata
I20260808 22:54:59.719612 4327 ts_tablet_manager.cc:1939] T 34ae651c379a48f5b45856fcbe42d0fc P 2cb3716c9dce4f00b783cfde2da4127b: Deleting consensus metadata
I20260808 22:54:59.719712 4328 ts_tablet_manager.cc:1939] T 34ae651c379a48f5b45856fcbe42d0fc P 2a4a08d4253a48d4af892fabaa3f3cf8: Deleting consensus metadata
I20260808 22:54:59.720005 3698 catalog_manager.cc:4841] TS 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533): tablet 34ae651c379a48f5b45856fcbe42d0fc (table test-table [id=cba998ca05094e978c3c9f180608aebf]) successfully deleted
I20260808 22:54:59.720141 3699 catalog_manager.cc:4841] TS 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577): tablet 34ae651c379a48f5b45856fcbe42d0fc (table test-table [id=cba998ca05094e978c3c9f180608aebf]) successfully deleted
I20260808 22:54:59.720288 3698 catalog_manager.cc:4841] TS 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355): tablet 34ae651c379a48f5b45856fcbe42d0fc (table test-table [id=cba998ca05094e978c3c9f180608aebf]) successfully deleted
I20260808 22:54:59.721146 4003 tablet_service.cc:1467] Processing CreateTablet for tablet b530696fedbe466ea0e6a00b689ef95c (DEFAULT_TABLE table=test-table [id=19cb282e77854727a605613edea0e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.721127 3857 tablet_service.cc:1467] Processing CreateTablet for tablet b530696fedbe466ea0e6a00b689ef95c (DEFAULT_TABLE table=test-table [id=19cb282e77854727a605613edea0e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.721151 4151 tablet_service.cc:1467] Processing CreateTablet for tablet b530696fedbe466ea0e6a00b689ef95c (DEFAULT_TABLE table=test-table [id=19cb282e77854727a605613edea0e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:54:59.721261 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b530696fedbe466ea0e6a00b689ef95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.721261 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b530696fedbe466ea0e6a00b689ef95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.721292 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b530696fedbe466ea0e6a00b689ef95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:54:59.722544 4338 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Bootstrap starting.
I20260808 22:54:59.722551 4341 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8: Bootstrap starting.
I20260808 22:54:59.722565 4340 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Bootstrap starting.
I20260808 22:54:59.722873 4338 tablet_bootstrap.cc:654] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.722947 4340 tablet_bootstrap.cc:654] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.722947 4341 tablet_bootstrap.cc:654] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8: Neither blocks nor log segments found. Creating new log.
I20260808 22:54:59.723511 4338 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: No bootstrap required, opened a new log
I20260808 22:54:59.723587 4338 ts_tablet_manager.cc:1397] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.723768 4340 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: No bootstrap required, opened a new log
I20260808 22:54:59.723733 4338 raft_consensus.cc:348] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.723824 4338 raft_consensus.cc:374] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.723842 4340 ts_tablet_manager.cc:1397] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:54:59.723837 4338 raft_consensus.cc:729] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5359149cc7430785d1727c583b1639, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.723904 4338 consensus_queue.cc:260] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.724010 4338 ts_tablet_manager.cc:1428] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:54:59.724017 4340 raft_consensus.cc:348] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.724102 4340 raft_consensus.cc:374] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.724123 4340 raft_consensus.cc:729] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.724172 4340 consensus_queue.cc:260] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.724277 4340 ts_tablet_manager.cc:1428] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:54:59.724510 4341 tablet_bootstrap.cc:492] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8: No bootstrap required, opened a new log
I20260808 22:54:59.724571 4341 ts_tablet_manager.cc:1397] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:54:59.724696 4341 raft_consensus.cc:348] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.724802 4341 raft_consensus.cc:374] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:54:59.724828 4341 raft_consensus.cc:729] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4a08d4253a48d4af892fabaa3f3cf8, State: Initialized, Role: FOLLOWER
I20260808 22:54:59.724875 4341 consensus_queue.cc:260] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.725032 4341 ts_tablet_manager.cc:1428] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:54:59.770550 4345 raft_consensus.cc:482] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:54:59.770669 4345 raft_consensus.cc:504] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.771005 4345 leader_election.cc:288] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355), 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533)
I20260808 22:54:59.775698 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.771459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:53571 (local address 127.1.212.1:38355)
0808 22:54:59.771538 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.771540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.771559 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:54:59.771716 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.771718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.771734 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.771798 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:59.772002 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.772845 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.774593 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.775211 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.775301 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.775317 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.775564 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.775619 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:54:59.775699 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.771634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:45629 (local address 127.1.212.2:35533)
0808 22:54:59.771784 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.771786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.771795 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:54:59.771857 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.771858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.771867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.771936 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:54:59.772090 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.772851 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.774594 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.775210 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.775308 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.775318 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.775561 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.775625 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260808 22:54:59.775709 4356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.771548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35533 (local address 127.1.212.3:45629)
0808 22:54:59.771729 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:59.771739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:59.771793 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:59.771968 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:59.771969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:59.771971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:59.772011 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.772012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.772963 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.772965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.774441 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.774446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.775338 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.775339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.775429 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.775433 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:54:59.775505 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:54:59.775565 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":113,"thread_start_us":116,"threads_started":1}
I20260808 22:54:59.776084 4026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b530696fedbe466ea0e6a00b689ef95c" candidate_uuid: "2cb3716c9dce4f00b783cfde2da4127b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639" is_pre_election: true
I20260808 22:54:59.776198 4026 raft_consensus.cc:2454] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb3716c9dce4f00b783cfde2da4127b in term 0.
I20260808 22:54:59.775705 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.771281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.1.212.3:53571)
0808 22:54:59.771467 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:59.771483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:59.771583 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:59.771806 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:59.771807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:59.771810 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:54:59.771888 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.771890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.772963 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.772965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.774442 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.774446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.775337 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.775339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.775425 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.775431 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:54:59.775505 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:54:59.775565 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":64,"thread_start_us":98,"threads_started":1}
I20260808 22:54:59.776439 4101 leader_election.cc:302] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb3716c9dce4f00b783cfde2da4127b, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:59.776618 4345 raft_consensus.cc:2781] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:54:59.776674 4345 raft_consensus.cc:482] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:54:59.776697 4345 raft_consensus.cc:3037] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.776764 3880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b530696fedbe466ea0e6a00b689ef95c" candidate_uuid: "2cb3716c9dce4f00b783cfde2da4127b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" is_pre_election: true
I20260808 22:54:59.776917 3880 raft_consensus.cc:2454] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb3716c9dce4f00b783cfde2da4127b in term 0.
I20260808 22:54:59.777626 4345 raft_consensus.cc:504] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.777803 4345 leader_election.cc:288] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [CANDIDATE]: Term 1 election: Requested vote from peers 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355), 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533)
I20260808 22:54:59.778019 3880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b530696fedbe466ea0e6a00b689ef95c" candidate_uuid: "2cb3716c9dce4f00b783cfde2da4127b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8"
I20260808 22:54:59.778054 4026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b530696fedbe466ea0e6a00b689ef95c" candidate_uuid: "2cb3716c9dce4f00b783cfde2da4127b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5359149cc7430785d1727c583b1639"
I20260808 22:54:59.778123 3880 raft_consensus.cc:3037] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.778153 4026 raft_consensus.cc:3037] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:54:59.778818 3880 raft_consensus.cc:2454] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cb3716c9dce4f00b783cfde2da4127b in term 1.
I20260808 22:54:59.778828 4026 raft_consensus.cc:2454] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cb3716c9dce4f00b783cfde2da4127b in term 1.
I20260808 22:54:59.779029 4101 leader_election.cc:302] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb3716c9dce4f00b783cfde2da4127b, 5c5359149cc7430785d1727c583b1639; no voters:
I20260808 22:54:59.779137 4345 raft_consensus.cc:2781] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:54:59.779304 4345 raft_consensus.cc:686] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 1 LEADER]: Becoming Leader. State: Replica: 2cb3716c9dce4f00b783cfde2da4127b, State: Running, Role: LEADER
I20260808 22:54:59.779392 4345 consensus_queue.cc:237] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } }
I20260808 22:54:59.780166 3711 catalog_manager.cc:5495] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3). New cstate: current_term: 1 leader_uuid: "2cb3716c9dce4f00b783cfde2da4127b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb3716c9dce4f00b783cfde2da4127b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39577 } health_report { overall_health: HEALTHY } } }
I20260808 22:54:59.802134 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.799357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.1.212.3:39577)
0808 22:54:59.799574 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.799576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.799582 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:54:59.799621 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.799621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.799643 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.799742 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:59.799886 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.800413 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.801250 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.801473 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.801555 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.801729 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.801856 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.801947 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.801988 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.802012 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":142,"threads_started":1}
I20260808 22:54:59.807101 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.804819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:56291 (local address 127.1.212.1:38355)
0808 22:54:59.804865 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.804867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.805007 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:54:59.805066 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.805067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.805082 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.805131 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:59.805260 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.805722 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.806451 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.806654 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.806711 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.806888 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.806918 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.806956 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.807008 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.807037 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:54:59.807116 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.804718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.1.212.3:56291)
0808 22:54:59.804928 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:59.804941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:59.805003 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:59.805147 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:59.805147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:59.805149 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:59.805219 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.805220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.805783 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.805784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.806362 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.806364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.806743 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.806744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.806796 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.806812 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.806970 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.806972 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:54:59.807006 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 22:54:59.807043 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":123,"threads_started":1}
I20260808 22:54:59.809686 4026 raft_consensus.cc:1261] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Refusing update from remote peer 2cb3716c9dce4f00b783cfde2da4127b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:59.809696 3880 raft_consensus.cc:1261] T b530696fedbe466ea0e6a00b689ef95c P 2a4a08d4253a48d4af892fabaa3f3cf8 [term 1 FOLLOWER]: Refusing update from remote peer 2cb3716c9dce4f00b783cfde2da4127b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:54:59.810029 4345 consensus_queue.cc:1035] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5359149cc7430785d1727c583b1639" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:59.810185 4358 consensus_queue.cc:1035] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4a08d4253a48d4af892fabaa3f3cf8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:54:59.818460 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.815872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:43465 (local address 127.1.212.3:39577)
0808 22:54:59.815946 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:54:59.815948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:54:59.816056 (+ 108us) server_negotiation.cc:407] Connection header received
0808 22:54:59.816122 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:54:59.816123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:54:59.816134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:54:59.816169 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:54:59.816282 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.816766 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.817894 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.818113 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.818172 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.818292 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.818320 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.818351 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 22:54:59.818384 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 22:54:59.818402 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:54:59.818481 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:54:59.815825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39577 (local address 127.1.212.1:43465)
0808 22:54:59.815994 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:54:59.816006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:54:59.816070 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:54:59.816178 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:54:59.816179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:54:59.816180 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:54:59.816242 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.816243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.816958 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.816959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.817816 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:54:59.817819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:54:59.818184 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:54:59.818185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:54:59.818237 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:54:59.818242 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:54:59.818352 (+ 110us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:54:59.818352 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 22:54:59.818383 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 22:54:59.818418 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":116,"threads_started":1}
W20260808 22:54:59.818948 4310 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:55:01.344718 3711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:01.344898 3711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:01.345834 3711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d80c67729b43bea33559cf4566b058: Sending DeleteTablet for 3 replicas of tablet b530696fedbe466ea0e6a00b689ef95c
I20260808 22:55:01.346163 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet b530696fedbe466ea0e6a00b689ef95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:01 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43002
I20260808 22:55:01.346161 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet b530696fedbe466ea0e6a00b689ef95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:01 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:48860
I20260808 22:55:01.346192 4151 tablet_service.cc:1514] Processing DeleteTablet for tablet b530696fedbe466ea0e6a00b689ef95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:01 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:39968
I20260808 22:55:01.346431 4397 tablet_replica.cc:330] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: stopping tablet replica
I20260808 22:55:01.346439 4395 tablet_replica.cc:330] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: stopping tablet replica
I20260808 22:55:01.346490 4397 raft_consensus.cc:2229] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:01.346496 4395 raft_consensus.cc:2229] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:01.346541 4395 raft_consensus.cc:2258] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:01.346567 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3791
I20260808 22:55:01.346613 4397 raft_consensus.cc:2258] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:01.347421 4395 ts_tablet_manager.cc:1905] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:01.347468 4397 ts_tablet_manager.cc:1905] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:01.349874 4397 ts_tablet_manager.cc:1918] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:55:01.349875 4395 ts_tablet_manager.cc:1918] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:55:01.349932 4397 log.cc:1196] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/b530696fedbe466ea0e6a00b689ef95c
I20260808 22:55:01.349942 4395 log.cc:1196] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/b530696fedbe466ea0e6a00b689ef95c
I20260808 22:55:01.350127 4397 ts_tablet_manager.cc:1939] T b530696fedbe466ea0e6a00b689ef95c P 2cb3716c9dce4f00b783cfde2da4127b: Deleting consensus metadata
I20260808 22:55:01.350188 4395 ts_tablet_manager.cc:1939] T b530696fedbe466ea0e6a00b689ef95c P 5c5359149cc7430785d1727c583b1639: Deleting consensus metadata
I20260808 22:55:01.350520 3699 catalog_manager.cc:4841] TS 2cb3716c9dce4f00b783cfde2da4127b (127.1.212.3:39577): tablet b530696fedbe466ea0e6a00b689ef95c (table test-table [id=19cb282e77854727a605613edea0e837]) successfully deleted
I20260808 22:55:01.350538 3698 catalog_manager.cc:4841] TS 5c5359149cc7430785d1727c583b1639 (127.1.212.2:35533): tablet b530696fedbe466ea0e6a00b689ef95c (table test-table [id=19cb282e77854727a605613edea0e837]) successfully deleted
W20260808 22:55:01.357904 3698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:38355 (error 108)
I20260808 22:55:01.358170 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3937
I20260808 22:55:01.361250 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:01.359825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38355 (local address 127.0.0.1:50632)
0808 22:55:01.361097 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:01.361180 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:38355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1234,"thread_start_us":92,"threads_started":1}
W20260808 22:55:01.361465 3698 catalog_manager.cc:4584] TS 2a4a08d4253a48d4af892fabaa3f3cf8 (127.1.212.1:38355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b530696fedbe466ea0e6a00b689ef95c: Network error: Client connection negotiation failed: client connection to 127.1.212.1:38355: connect: Connection refused (error 111)
I20260808 22:55:01.366932 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4083
I20260808 22:55:01.374889 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 3679
2026-08-08T22:55:01Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5604 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:01 dist-test-slave-1x4g krb5kdc[4402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:01 dist-test-slave-1x4g krb5kdc[4402](info): set up 2 sockets
Aug 08 22:55:01 dist-test-slave-1x4g krb5kdc[4402](info): commencing operation
krb5kdc: starting...
W20260808 22:55:02.468223 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:02 dist-test-slave-1x4g krb5kdc[4402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:02.604539 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46011
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:02.691874 4416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:02.692093 4416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:02.692116 4416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:02.693353 4416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:02.693399 4416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:02.693413 4416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:02.693426 4416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:02.693446 4416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:02.695032 4416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:46011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:02.695309 4416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:02.695626 4416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:02.700888 4421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:02.700934 4422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:02.700925 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:02.701299 4416 server_base.cc:1034] running on GCE node
I20260808 22:55:02.701565 4416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:02.701917 4416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:02.703073 4416 hybrid_clock.cc:648] HybridClock initialized: now 1786229702703054 us; error 31 us; skew 500 ppm
Aug 08 22:55:02 dist-test-slave-1x4g krb5kdc[4402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:02.705900 4416 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:02.706416 4416 webserver.cc:460] Webserver started at http://127.1.212.62:39925/ using document root <none> and password file <none>
I20260808 22:55:02.706694 4416 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:02.706745 4416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:02.706959 4416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:02.708488 4416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "21812e312131414ba95b43ad175003c1"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "1a0334cd4be4f2f1fc7d002fc2d8508f"
server_key_iv: "e6091f91ca8059424edaf7f5e9706c46"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.708922 4416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "21812e312131414ba95b43ad175003c1"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "1a0334cd4be4f2f1fc7d002fc2d8508f"
server_key_iv: "e6091f91ca8059424edaf7f5e9706c46"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.710470 4416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:02.711369 4432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:02.711601 4416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:02.711716 4416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "21812e312131414ba95b43ad175003c1"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "1a0334cd4be4f2f1fc7d002fc2d8508f"
server_key_iv: "e6091f91ca8059424edaf7f5e9706c46"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.711803 4416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:02.727190 4416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:02.729789 4416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:02.729975 4416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:02.734402 4416 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:46011
I20260808 22:55:02.734444 4488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:46011 every 8 connection(s)
I20260808 22:55:02.734835 4416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:02.735848 4489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:02.738479 4489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1: Bootstrap starting.
I20260808 22:55:02.739164 4489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:02.739499 4489 log.cc:824] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:02.740279 4489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1: No bootstrap required, opened a new log
I20260808 22:55:02.741202 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4416
I20260808 22:55:02.741312 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:02.741504 1872 external_mini_cluster.cc:1428] Setting key 30291ee761ced8dbd6572a05e8f27aa5
I20260808 22:55:02.742652 4489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } }
I20260808 22:55:02.742842 4489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:02.742884 4489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21812e312131414ba95b43ad175003c1, State: Initialized, Role: FOLLOWER
I20260808 22:55:02.743074 4489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } }
I20260808 22:55:02.743165 4489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:02.743189 4489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:02.743243 4489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:02.744403 4489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } }
I20260808 22:55:02.744526 4489 leader_election.cc:302] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21812e312131414ba95b43ad175003c1; no voters:
I20260808 22:55:02.744690 4489 leader_election.cc:288] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:02.744779 4494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:02.744966 4489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:02.745028 4494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [term 1 LEADER]: Becoming Leader. State: Replica: 21812e312131414ba95b43ad175003c1, State: Running, Role: LEADER
I20260808 22:55:02.745195 4494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } }
I20260808 22:55:02.745676 4496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21812e312131414ba95b43ad175003c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } } }
I20260808 22:55:02.745661 4494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21812e312131414ba95b43ad175003c1. Latest consensus state: current_term: 1 leader_uuid: "21812e312131414ba95b43ad175003c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21812e312131414ba95b43ad175003c1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46011 } } }
I20260808 22:55:02.745772 4496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:02.745779 4494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:02.746086 4503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:02.746742 4503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:02.749069 4503 catalog_manager.cc:1347] Generated new cluster ID: e9353d1b479347788fbb56a60e9e6236
I20260808 22:55:02.749130 4503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:55:02 dist-test-slave-1x4g krb5kdc[4402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:02.754797 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:02.742107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.1.212.62:46011)
0808 22:55:02.742343 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:02.742355 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:02.742395 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:55:02.742600 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:02.742608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:02.742672 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:02.742826 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:02.743728 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.745180 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.746406 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.747143 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.747253 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:02.750376 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:02.750381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:02.750381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:02.750396 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:02.752611 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:02.753138 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:02.753143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:02.753144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:02.753290 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:02.753528 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:02.753529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:02.753529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:02.753901 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:02.754131 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:02.754230 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:55:02.754556 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260808 22:55:02.758302 4503 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:02.759039 4503 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:02.783803 4503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21812e312131414ba95b43ad175003c1: Generated new TSK 0
I20260808 22:55:02.784063 4503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:02.820525 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:02.912348 4515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:02.912580 4515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:02.912604 4515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:02.913758 4515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:02.913797 4515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:02.913823 4515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:02.915387 4515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:02.915653 4515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:02.915913 4515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:02.921253 4524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:02.921263 4522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:02.921293 4521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:02.921731 4515 server_base.cc:1034] running on GCE node
I20260808 22:55:02.921960 4515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:02.922195 4515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:02.923380 4515 hybrid_clock.cc:648] HybridClock initialized: now 1786229702923362 us; error 36 us; skew 500 ppm
Aug 08 22:55:02 dist-test-slave-1x4g krb5kdc[4402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:02.926213 4515 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:02.926816 4515 webserver.cc:460] Webserver started at http://127.1.212.1:36099/ using document root <none> and password file <none>
I20260808 22:55:02.927109 4515 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:02.927191 4515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:02.927362 4515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:02.928601 4515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "dff9f029ca0847f3bc7caf9225bae2ca"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "9a40b36c4beff536012bbf0bca15ad12"
server_key_iv: "e8d3543a327532b27e77317464fcf7e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.929070 4515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "dff9f029ca0847f3bc7caf9225bae2ca"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "9a40b36c4beff536012bbf0bca15ad12"
server_key_iv: "e8d3543a327532b27e77317464fcf7e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.930604 4515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:02.931497 4532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:02.931695 4515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:02.931823 4515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "dff9f029ca0847f3bc7caf9225bae2ca"
format_stamp: "Formatted at 2026-08-08 22:55:02 on dist-test-slave-1x4g"
server_key: "9a40b36c4beff536012bbf0bca15ad12"
server_key_iv: "e8d3543a327532b27e77317464fcf7e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:02.931912 4515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:02.950630 4515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:02.953684 4515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:02.953879 4515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:02.954165 4515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:02.954530 4515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:02.954588 4515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:02.954638 4515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:02.954670 4515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:02.961288 4515 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:41489
I20260808 22:55:02.961347 4656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:41489 every 8 connection(s)
I20260808 22:55:02.961740 4515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:02.967036 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4515
I20260808 22:55:02.967170 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:02.967413 1872 external_mini_cluster.cc:1428] Setting key b06a994661c5df1c2b019521e03f8738
Aug 08 22:55:02 dist-test-slave-1x4g krb5kdc[4402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:02.973199 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:02.962636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:47797 (local address 127.1.212.62:46011)
0808 22:55:02.962711 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:02.962713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:02.962944 (+ 231us) server_negotiation.cc:407] Connection header received
0808 22:55:02.963479 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:02.963480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:02.963495 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:02.963559 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:02.964674 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.965341 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.966402 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.966751 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.966850 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:02.969588 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:02.969590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:02.969590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:02.969607 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:02.971324 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:02.971901 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:02.971902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:02.971903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:02.971971 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:02.972315 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:02.972316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:02.972317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:02.972473 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:02.972538 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:02.972988 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 22:55:02.973115 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:55:02.973462 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:02.962604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46011 (local address 127.1.212.1:47797)
0808 22:55:02.962805 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:02.962851 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:02.963379 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:02.963633 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:02.963637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:02.964287 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:02.964524 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:02.964529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.965456 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.965457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:02.966308 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:02.966310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:02.966862 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:02.966863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:02.966987 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:02.967715 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:02.967727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:02.969469 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:02.971424 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:02.971434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:02.971437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:02.971796 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:02.972053 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:02.972054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:02.972054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:02.972218 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:02.972552 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:02.972560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:02.972799 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 22:55:02.972934 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 22:55:02.973160 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
I20260808 22:55:02.974080 4657 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46011
I20260808 22:55:02.974253 4657 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:02.974502 4657 heartbeater.cc:511] Master 127.1.212.62:46011 requested a full tablet report, sending...
I20260808 22:55:02.975244 4450 ts_manager.cc:194] Registered new tserver with Master: dff9f029ca0847f3bc7caf9225bae2ca (127.1.212.1:41489)
I20260808 22:55:02.976245 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:47797
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:02.999758 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:03.086652 4662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:03.086894 4662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:03.086921 4662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:03.088233 4662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:03.088284 4662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:03.088312 4662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:03.089949 4662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:03.090247 4662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:03.090510 4662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:03.095875 4667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:03.095883 4668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:03.095875 4670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:03.096287 4662 server_base.cc:1034] running on GCE node
I20260808 22:55:03.096562 4662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:03.096908 4662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:03.098066 4662 hybrid_clock.cc:648] HybridClock initialized: now 1786229703098050 us; error 34 us; skew 500 ppm
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:03.100850 4662 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:03.101385 4662 webserver.cc:460] Webserver started at http://127.1.212.2:34177/ using document root <none> and password file <none>
I20260808 22:55:03.101662 4662 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:03.101712 4662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:03.101864 4662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:03.103017 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "624e98802f8d4931a367531a8a222515"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "2614cfbd3fcbfd0aef1a6d7bf961cc3b"
server_key_iv: "7432d6b9a9448631e2d10bc10ed20366"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.103425 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "624e98802f8d4931a367531a8a222515"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "2614cfbd3fcbfd0aef1a6d7bf961cc3b"
server_key_iv: "7432d6b9a9448631e2d10bc10ed20366"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.105024 4662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:03.105850 4678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.106051 4662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:03.106178 4662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "624e98802f8d4931a367531a8a222515"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "2614cfbd3fcbfd0aef1a6d7bf961cc3b"
server_key_iv: "7432d6b9a9448631e2d10bc10ed20366"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.106266 4662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:03.128346 4662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:03.130576 4662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:03.130760 4662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:03.131022 4662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:03.131366 4662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:03.131433 4662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.131474 4662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:03.131522 4662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.137895 4662 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:36127
I20260808 22:55:03.137953 4800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:36127 every 8 connection(s)
I20260808 22:55:03.138320 4662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:03.146756 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4662
I20260808 22:55:03.146852 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:03.147059 1872 external_mini_cluster.cc:1428] Setting key 0c3ee59715e1d720c5304751d34be611
I20260808 22:55:03.148592 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:03.139127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:39073 (local address 127.1.212.62:46011)
0808 22:55:03.139219 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:03.139222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:03.139415 (+ 193us) server_negotiation.cc:407] Connection header received
0808 22:55:03.139967 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:03.139969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:03.139982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:03.140033 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:03.141116 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.141640 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.142677 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.142905 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.142986 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:03.145295 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:03.145297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:03.145297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:03.145305 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:03.146798 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:03.147324 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:03.147326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:03.147326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:03.147393 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:03.147877 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:03.147878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:03.147879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:03.148007 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:03.148083 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:03.148393 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 22:55:03.148497 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:03.148875 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:03.139084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46011 (local address 127.1.212.2:39073)
0808 22:55:03.139275 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:03.139324 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:03.139854 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:03.140118 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:03.140123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:03.140725 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:03.141017 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:03.141022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.141744 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.141745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:03.142578 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:03.142581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.142995 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.142996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:03.143139 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:03.143627 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:03.143635 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:03.145193 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:03.146909 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:03.146919 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:03.146921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:03.147244 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:03.147672 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:03.147673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:03.147673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:03.147793 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:03.148086 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:03.148094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:03.148250 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 22:55:03.148370 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 22:55:03.148556 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260808 22:55:03.149361 4801 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46011
I20260808 22:55:03.149489 4801 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:03.149760 4801 heartbeater.cc:511] Master 127.1.212.62:46011 requested a full tablet report, sending...
I20260808 22:55:03.150213 4450 ts_manager.cc:194] Registered new tserver with Master: 624e98802f8d4931a367531a8a222515 (127.1.212.2:36127)
I20260808 22:55:03.150612 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:39073
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:03.176056 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:03.264683 4806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:03.265008 4806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:03.265038 4806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:03.266251 4806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:03.266294 4806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:03.266321 4806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:03.267923 4806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:03.268154 4806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:03.268362 4806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:03.274199 4812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:03.274233 4811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:03.274291 4806 server_base.cc:1034] running on GCE node
W20260808 22:55:03.274420 4814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:03.274660 4806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:03.274940 4806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:03.276125 4806 hybrid_clock.cc:648] HybridClock initialized: now 1786229703276119 us; error 47 us; skew 500 ppm
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:03.279157 4806 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:03.279724 4806 webserver.cc:460] Webserver started at http://127.1.212.3:44511/ using document root <none> and password file <none>
I20260808 22:55:03.279985 4806 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:03.280061 4806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:03.280225 4806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:03.281468 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "de0538ef3d534a518a89b29a36d1289e"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "bbae6d274745a9541d7f36f2435d785d"
server_key_iv: "7ec211469dde8fc003d3a55802d0a9e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.281880 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "de0538ef3d534a518a89b29a36d1289e"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "bbae6d274745a9541d7f36f2435d785d"
server_key_iv: "7ec211469dde8fc003d3a55802d0a9e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.283515 4806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:03.284562 4823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.284761 4806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:03.284920 4806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "de0538ef3d534a518a89b29a36d1289e"
format_stamp: "Formatted at 2026-08-08 22:55:03 on dist-test-slave-1x4g"
server_key: "bbae6d274745a9541d7f36f2435d785d"
server_key_iv: "7ec211469dde8fc003d3a55802d0a9e7"
server_key_version: "encryptionkey@0"
I20260808 22:55:03.285020 4806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:03.298233 4806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:03.300443 4806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:03.300623 4806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:03.300922 4806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:03.301272 4806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:03.301306 4806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.301360 4806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:03.301388 4806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:03.307698 4806 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:33119
I20260808 22:55:03.307778 4945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:33119 every 8 connection(s)
I20260808 22:55:03.308148 4806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:03.312867 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4806
I20260808 22:55:03.313014 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:03.313210 1872 external_mini_cluster.cc:1428] Setting key 9184470d6d6f837e37551cd869775277
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:03.318965 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:03.309067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:35787 (local address 127.1.212.62:46011)
0808 22:55:03.309123 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:03.309126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:03.309420 (+ 294us) server_negotiation.cc:407] Connection header received
0808 22:55:03.309937 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:03.309938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:03.309951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:03.310027 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:03.311113 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.311624 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.312716 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.313020 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.313090 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:03.315769 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:03.315771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:03.315771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:03.315780 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:03.317435 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:03.317954 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:03.317955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:03.317955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:03.318013 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:03.318335 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:03.318336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:03.318336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:03.318458 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:03.318511 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:03.318826 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 22:55:03.318897 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:55:03.319229 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:03.309056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46011 (local address 127.1.212.3:35787)
0808 22:55:03.309300 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:03.309340 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:03.309840 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:03.310086 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:03.310090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:03.310789 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:03.311011 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:03.311017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.311732 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.311733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:03.312603 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:03.312606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:03.313137 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:03.313139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:03.313309 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:03.314016 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:03.314028 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:03.315654 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:03.317526 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:03.317536 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:03.317539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:03.317867 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:03.318102 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:03.318104 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:03.318104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:03.318265 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:03.318529 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:03.318536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:03.318692 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 22:55:03.318796 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 22:55:03.318952 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
I20260808 22:55:03.319810 4946 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46011
I20260808 22:55:03.319969 4946 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:03.320195 4946 heartbeater.cc:511] Master 127.1.212.62:46011 requested a full tablet report, sending...
I20260808 22:55:03.320679 4450 ts_manager.cc:194] Registered new tserver with Master: de0538ef3d534a518a89b29a36d1289e (127.1.212.3:33119)
I20260808 22:55:03.321144 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:35787
I20260808 22:55:03.325333 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:55:03.339941 4493 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:03.338821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58224 (local address 127.1.212.62:46011)
0808 22:55:03.338922 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:03.338924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:03.339008 (+ 84us) server_negotiation.cc:407] Connection header received
0808 22:55:03.339065 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:03.339066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:03.339079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:03.339138 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:03.339837 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58224: BlockingRecv error: recv got EOF from 127.0.0.1:58224 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:55:03.341516 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4515
I20260808 22:55:03.351791 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4662
I20260808 22:55:03.359333 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4806
I20260808 22:55:03.367375 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4416
2026-08-08T22:55:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1989 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4958](info): set up 2 sockets
Aug 08 22:55:03 dist-test-slave-1x4g krb5kdc[4958](info): commencing operation
krb5kdc: starting...
W20260808 22:55:04.496623 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[4958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:04.636377 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:41081
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:04.726090 4971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:04.726377 4971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:04.726424 4971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:04.727789 4971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:04.727886 4971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:04.727923 4971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:04.727957 4971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:04.727980 4971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:04.729914 4971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:04.730262 4971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:04.730540 4971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:04.736078 4979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:04.736065 4977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:04.736064 4976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:04.736099 4971 server_base.cc:1034] running on GCE node
I20260808 22:55:04.736557 4971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:04.736958 4971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:04.738119 4971 hybrid_clock.cc:648] HybridClock initialized: now 1786229704738107 us; error 33 us; skew 500 ppm
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[4958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229704, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:04.746023 4971 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:04.746559 4971 webserver.cc:460] Webserver started at http://127.1.212.62:45693/ using document root <none> and password file <none>
I20260808 22:55:04.746851 4971 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:04.746932 4971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:04.747104 4971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:04.748307 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "2b4d5549c95c409fb5f98bbe81b17b50"
format_stamp: "Formatted at 2026-08-08 22:55:04 on dist-test-slave-1x4g"
server_key: "9369b13252d8fc6b5a102d09b24b846b"
server_key_iv: "b8b8266fd653c44eb58d0948d87445cc"
server_key_version: "encryptionkey@0"
I20260808 22:55:04.748716 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "2b4d5549c95c409fb5f98bbe81b17b50"
format_stamp: "Formatted at 2026-08-08 22:55:04 on dist-test-slave-1x4g"
server_key: "9369b13252d8fc6b5a102d09b24b846b"
server_key_iv: "b8b8266fd653c44eb58d0948d87445cc"
server_key_version: "encryptionkey@0"
I20260808 22:55:04.750324 4971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:04.751226 4986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:04.751442 4971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:04.751577 4971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "2b4d5549c95c409fb5f98bbe81b17b50"
format_stamp: "Formatted at 2026-08-08 22:55:04 on dist-test-slave-1x4g"
server_key: "9369b13252d8fc6b5a102d09b24b846b"
server_key_iv: "b8b8266fd653c44eb58d0948d87445cc"
server_key_version: "encryptionkey@0"
I20260808 22:55:04.751667 4971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:04.762923 4971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:04.765250 4971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:04.765380 4971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:04.769824 4971 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:41081
I20260808 22:55:04.769867 5041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:41081 every 8 connection(s)
I20260808 22:55:04.770274 4971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:04.771250 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:04.773356 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 4971
I20260808 22:55:04.773478 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:04.773674 1872 external_mini_cluster.cc:1428] Setting key b9439b1878f2d641703a07239861ae41
I20260808 22:55:04.774190 5042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50: Bootstrap starting.
I20260808 22:55:04.774927 5042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:04.775267 5042 log.cc:824] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:04.776046 5042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50: No bootstrap required, opened a new log
I20260808 22:55:04.778122 5042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } }
I20260808 22:55:04.778312 5042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:04.778364 5042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4d5549c95c409fb5f98bbe81b17b50, State: Initialized, Role: FOLLOWER
I20260808 22:55:04.778577 5042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } }
I20260808 22:55:04.778695 5042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:04.778736 5042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:04.778800 5042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:04.779693 5042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } }
I20260808 22:55:04.779843 5042 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b4d5549c95c409fb5f98bbe81b17b50; no voters:
I20260808 22:55:04.780068 5042 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:04.780170 5048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:04.780408 5048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [term 1 LEADER]: Becoming Leader. State: Replica: 2b4d5549c95c409fb5f98bbe81b17b50, State: Running, Role: LEADER
I20260808 22:55:04.780421 5042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:04.780622 5048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } }
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[4958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:04.781235 5050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4d5549c95c409fb5f98bbe81b17b50. Latest consensus state: current_term: 1 leader_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } } }
I20260808 22:55:04.781322 5050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:04.781570 5058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:04.782177 5058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:04.782222 5049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4d5549c95c409fb5f98bbe81b17b50" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 41081 } } }
I20260808 22:55:04.783006 5049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:04.784345 5058 catalog_manager.cc:1347] Generated new cluster ID: 2a0077a841c04189ba4980fd750b661d
I20260808 22:55:04.784444 5058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:04.785703 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:04.774323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.1.212.62:41081)
0808 22:55:04.774547 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:04.774555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:04.774582 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:55:04.775110 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:04.775118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:04.775171 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:04.775323 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:04.776002 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:04.777073 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:04.777859 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:04.778306 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:04.778450 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:04.781786 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:04.781791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:04.781792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:04.781806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:04.783919 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:04.784423 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:04.784427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:04.784427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:04.784570 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:04.784810 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:04.784812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:04.784812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:04.785086 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:04.785262 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:04.785365 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 22:55:04.785517 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260808 22:55:04.786653 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:55:04.796118 5058 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:04.796731 5058 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:04.804566 5058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b4d5549c95c409fb5f98bbe81b17b50: Generated new TSK 0
I20260808 22:55:04.804833 5058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:04.811509 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:04.807483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.1.212.62:41081)
0808 22:55:04.807538 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:04.807540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:04.807662 (+ 122us) server_negotiation.cc:407] Connection header received
0808 22:55:04.807739 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:04.807740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:04.807749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:04.807806 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:04.808276 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:04.808884 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:04.809996 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:04.810230 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:04.810286 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:04.811030 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:04.811031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:04.811031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:04.811232 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:04.811249 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:04.811255 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:04.811302 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:04.811394 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 22:55:04.811422 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:55:04.814356 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 4971
2026-08-08T22:55:04Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1445 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[5078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[5078](info): set up 2 sockets
Aug 08 22:55:04 dist-test-slave-1x4g krb5kdc[5078](info): commencing operation
krb5kdc: starting...
W20260808 22:55:05.893074 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:05 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:06.028949 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40555
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:06.116343 5093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:06.116672 5093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:06.116726 5093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:06.118109 5093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:06.118196 5093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:06.118221 5093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:06.118245 5093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:06.118275 5093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:06.120018 5093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:06.120373 5093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:06.120638 5093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:06.126053 5099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.126080 5098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.126063 5101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.126353 5093 server_base.cc:1034] running on GCE node
I20260808 22:55:06.126637 5093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:06.127004 5093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:06.128185 5093 hybrid_clock.cc:648] HybridClock initialized: now 1786229706128171 us; error 48 us; skew 500 ppm
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:06.131048 5093 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:06.131604 5093 webserver.cc:460] Webserver started at http://127.1.212.62:39365/ using document root <none> and password file <none>
I20260808 22:55:06.131873 5093 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:06.131949 5093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:06.132110 5093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:06.133369 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "b5f5792e72ab45808e54127b0e93beba"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "c789ab1a3548002f18d7e4627ecc9130"
server_key_iv: "988fd76d78700a11fa4c2ef8ec1db4e4"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.133770 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "b5f5792e72ab45808e54127b0e93beba"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "c789ab1a3548002f18d7e4627ecc9130"
server_key_iv: "988fd76d78700a11fa4c2ef8ec1db4e4"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.135298 5093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 22:55:06.136152 5108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.136345 5093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:06.136471 5093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "b5f5792e72ab45808e54127b0e93beba"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "c789ab1a3548002f18d7e4627ecc9130"
server_key_iv: "988fd76d78700a11fa4c2ef8ec1db4e4"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.136559 5093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:06.161516 5093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:06.163722 5093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:06.163918 5093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:06.168435 5093 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:40555
I20260808 22:55:06.168478 5163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:40555 every 8 connection(s)
I20260808 22:55:06.168938 5093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:06.170217 5164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:06.172709 5164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba: Bootstrap starting.
I20260808 22:55:06.173416 5164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:06.173676 5164 log.cc:824] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:06.174448 5164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba: No bootstrap required, opened a new log
I20260808 22:55:06.175809 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5093
I20260808 22:55:06.175922 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:06.176110 1872 external_mini_cluster.cc:1428] Setting key eda381301f622a0532fdce4854e6bb1a
I20260808 22:55:06.176558 5164 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } }
I20260808 22:55:06.176808 5164 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:06.176857 5164 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f5792e72ab45808e54127b0e93beba, State: Initialized, Role: FOLLOWER
I20260808 22:55:06.177033 5164 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } }
I20260808 22:55:06.177148 5164 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:06.177229 5164 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:06.177290 5164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:06.178493 5164 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } }
I20260808 22:55:06.178611 5164 leader_election.cc:302] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5f5792e72ab45808e54127b0e93beba; no voters:
I20260808 22:55:06.178781 5164 leader_election.cc:288] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:06.178877 5169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:06.179064 5169 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [term 1 LEADER]: Becoming Leader. State: Replica: b5f5792e72ab45808e54127b0e93beba, State: Running, Role: LEADER
I20260808 22:55:06.179154 5164 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:06.179210 5169 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } }
I20260808 22:55:06.179688 5169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5f5792e72ab45808e54127b0e93beba. Latest consensus state: current_term: 1 leader_uuid: "b5f5792e72ab45808e54127b0e93beba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } } }
I20260808 22:55:06.179697 5171 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5f5792e72ab45808e54127b0e93beba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f5792e72ab45808e54127b0e93beba" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40555 } } }
I20260808 22:55:06.179853 5169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:06.179860 5171 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:06.180135 5180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:06.180708 5180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:06.182618 5180 catalog_manager.cc:1347] Generated new cluster ID: 4b2e9af076fa47e6b5225e77b61646c5
I20260808 22:55:06.182674 5180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:06.188905 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.176732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.1.212.62:40555)
0808 22:55:06.177398 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:06.177409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:06.177444 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:55:06.177582 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:06.177589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:06.177645 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:06.177812 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:06.178508 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.179941 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.181106 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.181648 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.181811 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.184646 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:06.184650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:06.184651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:06.184668 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:06.186704 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.187126 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.187130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.187131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.187252 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.187429 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.187430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.187430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.187744 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:06.187897 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:06.188444 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 22:55:06.188621 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":530,"threads_started":1}
I20260808 22:55:06.205710 5180 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:06.206322 5180 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:06.211676 5180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5f5792e72ab45808e54127b0e93beba: Generated new TSK 0
I20260808 22:55:06.211916 5180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:06.243754 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:06.331139 5193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:06.331369 5193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:06.331398 5193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:06.332720 5193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:06.332850 5193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:06.332927 5193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:06.334543 5193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:06.334834 5193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:06.335098 5193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:55:06.340535 5193 server_base.cc:1034] running on GCE node
W20260808 22:55:06.340579 5199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.340597 5198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.340682 5201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.340972 5193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:06.341320 5193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:06.342483 5193 hybrid_clock.cc:648] HybridClock initialized: now 1786229706342471 us; error 28 us; skew 500 ppm
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:06.345355 5193 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:06.345806 5193 webserver.cc:460] Webserver started at http://127.1.212.1:42267/ using document root <none> and password file <none>
I20260808 22:55:06.346009 5193 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:06.346051 5193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:06.346146 5193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:06.347225 5193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "fac5cba9faef4bff9257381ad8af508a"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "83cb0a893d615b50cb6f13d67655e5b6"
server_key_iv: "38b5fbe59ba21724acb4c36ddea98a03"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.347558 5193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "fac5cba9faef4bff9257381ad8af508a"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "83cb0a893d615b50cb6f13d67655e5b6"
server_key_iv: "38b5fbe59ba21724acb4c36ddea98a03"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.349009 5193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:06.349911 5208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.350139 5193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:06.350237 5193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "fac5cba9faef4bff9257381ad8af508a"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "83cb0a893d615b50cb6f13d67655e5b6"
server_key_iv: "38b5fbe59ba21724acb4c36ddea98a03"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.350304 5193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:06.359543 5193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:06.362094 5193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:06.362221 5193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:06.362437 5193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:06.362720 5193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:06.362754 5193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.362777 5193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:06.362790 5193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.368857 5193 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:34471
I20260808 22:55:06.368957 5328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:34471 every 8 connection(s)
I20260808 22:55:06.369401 5193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:06.370074 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5193
I20260808 22:55:06.370193 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:06.370427 1872 external_mini_cluster.cc:1428] Setting key a9e120a3174b717ae14539fc5c7fcf9c
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
I20260808 22:55:06.381453 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.370334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:45067 (local address 127.1.212.62:40555)
0808 22:55:06.370417 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:06.370420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:06.370651 (+ 231us) server_negotiation.cc:407] Connection header received
0808 22:55:06.371368 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:06.371369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:06.371383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:06.371444 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:06.372959 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.373512 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.374634 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.374848 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.374931 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.377803 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:06.377806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:06.377806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:06.377817 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:06.379735 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.380279 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.380281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.380281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.380350 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.380623 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.380624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.380624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.380817 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:06.380889 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:06.381226 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 22:55:06.381339 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:55:06.381573 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.370272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40555 (local address 127.1.212.1:45067)
0808 22:55:06.370477 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:06.370535 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:06.371232 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:06.371536 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:06.371544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:06.372477 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:06.372836 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.372845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.373634 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.373635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.374530 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.374534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.374946 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.374947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.375103 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.375701 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:06.375711 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:06.377679 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:06.379870 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.379882 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.379885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.380184 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.380432 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.380432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.380433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.380550 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.380881 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:06.380907 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:06.381076 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 22:55:06.381191 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:55:06.381342 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260808 22:55:06.382144 5329 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40555
I20260808 22:55:06.382308 5329 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:06.382519 5329 heartbeater.cc:511] Master 127.1.212.62:40555 requested a full tablet report, sending...
I20260808 22:55:06.383061 5126 ts_manager.cc:194] Registered new tserver with Master: fac5cba9faef4bff9257381ad8af508a (127.1.212.1:34471)
I20260808 22:55:06.383811 5126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:45067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:06.402202 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:06.489806 5336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:06.490051 5336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:06.490080 5336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:06.491329 5336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:06.491382 5336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:06.491412 5336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:06.492991 5336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:06.493225 5336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:06.493435 5336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:06.499070 5342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.499171 5336 server_base.cc:1034] running on GCE node
W20260808 22:55:06.499049 5344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.499049 5341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.499619 5336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:06.499886 5336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:06.501039 5336 hybrid_clock.cc:648] HybridClock initialized: now 1786229706501024 us; error 31 us; skew 500 ppm
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:06.503839 5336 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:06.504357 5336 webserver.cc:460] Webserver started at http://127.1.212.2:43253/ using document root <none> and password file <none>
I20260808 22:55:06.504609 5336 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:06.504683 5336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:06.504873 5336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:06.506031 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "1d16e11396874a579508b0132ac20d9e"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "0960883c4902c937b769e48ed0023658"
server_key_iv: "032eb1c92a46aa4fde09a202b1503eb3"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.506448 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "1d16e11396874a579508b0132ac20d9e"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "0960883c4902c937b769e48ed0023658"
server_key_iv: "032eb1c92a46aa4fde09a202b1503eb3"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.507922 5336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:06.508858 5351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.509049 5336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:06.509173 5336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "1d16e11396874a579508b0132ac20d9e"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "0960883c4902c937b769e48ed0023658"
server_key_iv: "032eb1c92a46aa4fde09a202b1503eb3"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.509265 5336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:06.517482 5336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:06.519491 5336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:06.519688 5336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:06.519954 5336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:06.520298 5336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:06.520355 5336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.520398 5336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:06.520442 5336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.526829 5336 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:33169
I20260808 22:55:06.526871 5470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:33169 every 8 connection(s)
I20260808 22:55:06.527244 5336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:06.528563 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5336
I20260808 22:55:06.528657 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:06.528908 1872 external_mini_cluster.cc:1428] Setting key 234aa2166328e31d9d43cea4fa281c72
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:06.538508 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.528031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46043 (local address 127.1.212.62:40555)
0808 22:55:06.528094 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:06.528096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:06.528310 (+ 214us) server_negotiation.cc:407] Connection header received
0808 22:55:06.529020 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:06.529022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:06.529038 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:06.529128 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:06.530259 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.531033 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.532134 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.532453 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.532547 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.535362 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:06.535364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:06.535364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:06.535373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:06.536926 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.537460 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.537461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.537461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.537516 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.537819 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.537820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.537820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.537935 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:06.537986 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:06.538362 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 22:55:06.538438 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:55:06.538836 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.527985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40555 (local address 127.1.212.2:46043)
0808 22:55:06.528177 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:06.528224 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:06.528879 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:06.529219 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:06.529223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:06.529923 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:06.530151 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.530156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.531140 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.531141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.532017 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.532020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.532576 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.532577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.532689 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.533367 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:06.533377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:06.535244 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:06.537022 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.537032 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.537035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.537382 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.537587 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.537588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.537589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.537747 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.538007 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:06.538015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:06.538208 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 22:55:06.538343 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 22:55:06.538554 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
I20260808 22:55:06.539541 5471 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40555
I20260808 22:55:06.539670 5471 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:06.539866 5471 heartbeater.cc:511] Master 127.1.212.62:40555 requested a full tablet report, sending...
I20260808 22:55:06.540293 5126 ts_manager.cc:194] Registered new tserver with Master: 1d16e11396874a579508b0132ac20d9e (127.1.212.2:33169)
I20260808 22:55:06.540755 5126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:46043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:06.560539 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:06.649046 5476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:06.649376 5476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:06.649422 5476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:06.650749 5476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:06.650826 5476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:06.650875 5476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:06.652496 5476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:06.652820 5476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:06.653071 5476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:06.658823 5481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:06.658811 5485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.658851 5476 server_base.cc:1034] running on GCE node
W20260808 22:55:06.658811 5482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:06.659329 5476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:06.659650 5476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:06.660844 5476 hybrid_clock.cc:648] HybridClock initialized: now 1786229706660816 us; error 37 us; skew 500 ppm
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:06.663682 5476 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:06.664224 5476 webserver.cc:460] Webserver started at http://127.1.212.3:45293/ using document root <none> and password file <none>
I20260808 22:55:06.664497 5476 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:06.664554 5476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:06.664734 5476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:06.665947 5476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "69d3d51565ca4c29af2ece9916b2a4c9"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "d04cfdf34f4d0fe0d71e9efccbcdfd91"
server_key_iv: "5f788a2944702e34199a270497a653aa"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.666349 5476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "69d3d51565ca4c29af2ece9916b2a4c9"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "d04cfdf34f4d0fe0d71e9efccbcdfd91"
server_key_iv: "5f788a2944702e34199a270497a653aa"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.667789 5476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:06.668593 5492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.668846 5476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:06.668936 5476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "69d3d51565ca4c29af2ece9916b2a4c9"
format_stamp: "Formatted at 2026-08-08 22:55:06 on dist-test-slave-1x4g"
server_key: "d04cfdf34f4d0fe0d71e9efccbcdfd91"
server_key_iv: "5f788a2944702e34199a270497a653aa"
server_key_version: "encryptionkey@0"
I20260808 22:55:06.669044 5476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:06.680302 5476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:06.682960 5476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:06.683133 5476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:06.683420 5476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:06.683784 5476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:06.683822 5476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.683869 5476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:06.683897 5476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:06.690603 5476 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:34809
I20260808 22:55:06.690668 5616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:34809 every 8 connection(s)
I20260808 22:55:06.691051 5476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:06.697186 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5476
I20260808 22:55:06.697276 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:06.697608 1872 external_mini_cluster.cc:1428] Setting key fa66d7d9656725cafd34b4d6e1e7d7bb
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:06.701246 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.691887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:45945 (local address 127.1.212.62:40555)
0808 22:55:06.691969 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:06.691972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:06.692163 (+ 191us) server_negotiation.cc:407] Connection header received
0808 22:55:06.692654 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:06.692655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:06.692666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:06.692720 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:06.693898 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.694410 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.695487 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.695690 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.695752 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.698526 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:06.698528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:06.698528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:06.698538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:06.699839 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.700282 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.700283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.700283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.700340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.700576 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.700577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.700578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.700690 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:06.700743 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:06.701097 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 22:55:06.701174 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:55:06.701435 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.691832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40555 (local address 127.1.212.3:45945)
0808 22:55:06.692037 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:06.692077 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:06.692569 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:06.692924 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:06.692929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:06.693551 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:06.693773 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.693779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.694508 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.694509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.695396 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:06.695398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.695777 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.695778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:06.695888 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.696510 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:06.696522 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:06.698377 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:06.699934 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.699943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.699945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.700208 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.700402 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:06.700403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:06.700403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:06.700519 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:06.700748 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:06.700752 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:06.700965 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 22:55:06.701061 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:55:06.701208 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260808 22:55:06.701985 5617 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40555
I20260808 22:55:06.702122 5617 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:06.702334 5617 heartbeater.cc:511] Master 127.1.212.62:40555 requested a full tablet report, sending...
I20260808 22:55:06.702795 5126 ts_manager.cc:194] Registered new tserver with Master: 69d3d51565ca4c29af2ece9916b2a4c9 (127.1.212.3:34809)
I20260808 22:55:06.703207 5126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:45945
I20260808 22:55:06.709649 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229706, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:06.736392 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:06.728342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.1.212.62:40555)
0808 22:55:06.728438 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:06.728441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:06.728547 (+ 106us) server_negotiation.cc:407] Connection header received
0808 22:55:06.728669 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:06.728670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:06.728689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:06.728757 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:06.729451 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.729972 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.730759 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:06.730971 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:06.731057 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:06.733566 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:06.733568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:06.733568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:06.733577 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:06.735028 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.735448 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.735449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.735450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.735512 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:06.735718 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:06.735719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:06.735719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:06.735895 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:06.736019 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:06.736174 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 22:55:06.736271 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 22:55:06.736675 5126 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35116
I20260808 22:55:06.737222 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5193
I20260808 22:55:06.746001 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5336
I20260808 22:55:06.752543 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5476
I20260808 22:55:06.760528 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5093
2026-08-08T22:55:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1947 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5630](info): set up 2 sockets
Aug 08 22:55:06 dist-test-slave-1x4g krb5kdc[5630](info): commencing operation
krb5kdc: starting...
W20260808 22:55:07.840356 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:07 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:07.980103 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:44493
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:44493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:08.067754 5646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:08.068001 5646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:08.068032 5646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:08.069337 5646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:08.069386 5646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:08.069401 5646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:08.069413 5646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:08.069482 5646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:08.071146 5646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:44493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:44493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:08.071455 5646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:08.071724 5646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:08.077111 5651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.077251 5646 server_base.cc:1034] running on GCE node
W20260808 22:55:08.077219 5653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:08.077353 5655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.077693 5646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:08.077996 5646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:08.079167 5646 hybrid_clock.cc:648] HybridClock initialized: now 1786229708079149 us; error 31 us; skew 500 ppm
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:08.082103 5646 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:08.082612 5646 webserver.cc:460] Webserver started at http://127.1.212.62:46659/ using document root <none> and password file <none>
I20260808 22:55:08.082883 5646 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:08.082959 5646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:08.083118 5646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:08.084280 5646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "97a07ab2437a41bd99688e049b105bfb"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "aeaad6864e858ff90395a7e7abaffebf"
server_key_iv: "a9e6d2f3c268779823b8cd75b14097f9"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.084692 5646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "97a07ab2437a41bd99688e049b105bfb"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "aeaad6864e858ff90395a7e7abaffebf"
server_key_iv: "a9e6d2f3c268779823b8cd75b14097f9"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.086228 5646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:08.087137 5662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.087337 5646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:08.087467 5646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "97a07ab2437a41bd99688e049b105bfb"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "aeaad6864e858ff90395a7e7abaffebf"
server_key_iv: "a9e6d2f3c268779823b8cd75b14097f9"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.087555 5646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:08.096297 5646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:08.098727 5646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:08.098920 5646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:08.103396 5646 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:44493
I20260808 22:55:08.103451 5719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:44493 every 8 connection(s)
I20260808 22:55:08.104130 5646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:08.105093 5720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:08.106704 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5646
I20260808 22:55:08.106833 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:08.107004 1872 external_mini_cluster.cc:1428] Setting key 8480fcac64afa5d329bf8dcd8185d495
I20260808 22:55:08.108300 5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb: Bootstrap starting.
I20260808 22:55:08.109118 5720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:08.109467 5720 log.cc:824] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:08.110417 5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb: No bootstrap required, opened a new log
I20260808 22:55:08.112957 5720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } }
I20260808 22:55:08.113169 5720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:08.113212 5720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a07ab2437a41bd99688e049b105bfb, State: Initialized, Role: FOLLOWER
I20260808 22:55:08.113358 5720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } }
I20260808 22:55:08.113458 5720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:08.113535 5720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:08.113596 5720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:08.114282 5720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } }
I20260808 22:55:08.114390 5720 leader_election.cc:302] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97a07ab2437a41bd99688e049b105bfb; no voters:
I20260808 22:55:08.114665 5720 leader_election.cc:288] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:08.114698 5725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:08.114816 5725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [term 1 LEADER]: Becoming Leader. State: Replica: 97a07ab2437a41bd99688e049b105bfb, State: Running, Role: LEADER
I20260808 22:55:08.114921 5725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } }
I20260808 22:55:08.114989 5720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:08.115279 5725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97a07ab2437a41bd99688e049b105bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } } }
I20260808 22:55:08.115473 5725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:08.115702 5725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a07ab2437a41bd99688e049b105bfb. Latest consensus state: current_term: 1 leader_uuid: "97a07ab2437a41bd99688e049b105bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a07ab2437a41bd99688e049b105bfb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 44493 } } }
I20260808 22:55:08.115849 5725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:08.116333 5733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:08.116984 5733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:08.119058 5733 catalog_manager.cc:1347] Generated new cluster ID: d2a13606bc8645ad9b9a9ce3b900b3b3
I20260808 22:55:08.119123 5733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:08.120287 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.107643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.1.212.62:44493)
0808 22:55:08.107867 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.107878 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.107915 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:55:08.108509 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.108520 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.108587 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.108757 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.109575 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.110978 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.111794 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.112568 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.112710 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.115947 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.115953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.115953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.115976 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.118290 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.118815 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.118820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.118820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.118942 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.119209 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.119210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.119210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.119534 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.119682 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.119883 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.120043 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260808 22:55:08.125087 5733 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:08.125713 5733 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:08.128870 5733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a07ab2437a41bd99688e049b105bfb: Generated new TSK 0
I20260808 22:55:08.129123 5733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:08.160524 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:08.249344 5747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:08.249610 5747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:08.249655 5747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:08.250885 5747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:08.250965 5747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:08.251013 5747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:08.252652 5747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:08.252982 5747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:08.253247 5747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:08.258646 5755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:08.258680 5754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.258729 5747 server_base.cc:1034] running on GCE node
W20260808 22:55:08.258653 5757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.259227 5747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:08.259460 5747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:08.260661 5747 hybrid_clock.cc:648] HybridClock initialized: now 1786229708260650 us; error 45 us; skew 500 ppm
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:08.263577 5747 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:08.264133 5747 webserver.cc:460] Webserver started at http://127.1.212.1:45829/ using document root <none> and password file <none>
I20260808 22:55:08.264379 5747 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:08.264456 5747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:08.264616 5747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:08.265854 5747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "7b13103ad8184c7fb7c56129e557b065"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "fc4e65d7e00a45fe82040db4b6a59c1e"
server_key_iv: "a481d206541d592aa37e166619c354dc"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.266256 5747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "7b13103ad8184c7fb7c56129e557b065"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "fc4e65d7e00a45fe82040db4b6a59c1e"
server_key_iv: "a481d206541d592aa37e166619c354dc"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.267699 5747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:08.268496 5765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.268705 5747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:08.268847 5747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "7b13103ad8184c7fb7c56129e557b065"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "fc4e65d7e00a45fe82040db4b6a59c1e"
server_key_iv: "a481d206541d592aa37e166619c354dc"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.268950 5747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:08.280383 5747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:08.282987 5747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:08.283166 5747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:08.283442 5747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:08.283800 5747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:08.283861 5747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.283917 5747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:08.283947 5747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.290740 5747 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:41421
I20260808 22:55:08.290827 5890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:41421 every 8 connection(s)
I20260808 22:55:08.291239 5747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:08.297531 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5747
I20260808 22:55:08.297681 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:08.297904 1872 external_mini_cluster.cc:1428] Setting key d6644ffdca206fd4a82e279e9c8fb634
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:08.302074 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.292137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:52775 (local address 127.1.212.62:44493)
0808 22:55:08.292185 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.292187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.292416 (+ 229us) server_negotiation.cc:407] Connection header received
0808 22:55:08.293597 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.293599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.293614 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.293673 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.294391 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.294937 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.296020 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.296237 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.296320 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.298934 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.298936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.298936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.298945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.300401 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.300973 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.300974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.300975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.301047 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.301335 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.301336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.301337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.301469 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.301549 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.301910 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.302001 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:55:08.302356 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.292100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:44493 (local address 127.1.212.1:52775)
0808 22:55:08.292270 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:08.292313 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:08.293464 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:08.293756 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:08.293761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:08.294067 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:08.294278 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.294283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.295055 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.295056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.295919 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.295921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.296333 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.296334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.296480 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.297093 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:08.297102 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:08.298817 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:08.300543 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.300553 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.300555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.300880 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.301124 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.301125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.301125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.301257 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.301538 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:08.301547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:08.301739 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 22:55:08.301854 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:55:08.302053 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260808 22:55:08.302977 5891 heartbeater.cc:347] Connected to a master server at 127.1.212.62:44493
I20260808 22:55:08.303088 5891 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:08.303311 5891 heartbeater.cc:511] Master 127.1.212.62:44493 requested a full tablet report, sending...
I20260808 22:55:08.303952 5679 ts_manager.cc:194] Registered new tserver with Master: 7b13103ad8184c7fb7c56129e557b065 (127.1.212.1:41421)
I20260808 22:55:08.304719 5679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:52775
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:08.328622 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:08.418486 5896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:08.418774 5896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:08.418821 5896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:08.420197 5896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:08.420280 5896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:08.420329 5896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:08.422107 5896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:08.422426 5896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:08.422685 5896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:08.428145 5902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:08.428104 5903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.428332 5896 server_base.cc:1034] running on GCE node
W20260808 22:55:08.428104 5905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.428630 5896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:08.428997 5896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:08.430162 5896 hybrid_clock.cc:648] HybridClock initialized: now 1786229708430144 us; error 31 us; skew 500 ppm
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:08.432924 5896 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:08.433418 5896 webserver.cc:460] Webserver started at http://127.1.212.2:41895/ using document root <none> and password file <none>
I20260808 22:55:08.433676 5896 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:08.433749 5896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:08.433909 5896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:08.435022 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "e92966c15b754324a91025b430e79b38"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3b8127eefb6774661ae3f2b990693fa6"
server_key_iv: "5c8b7096cba1f63ca7285247fa395334"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.435416 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "e92966c15b754324a91025b430e79b38"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3b8127eefb6774661ae3f2b990693fa6"
server_key_iv: "5c8b7096cba1f63ca7285247fa395334"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.436916 5896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:08.437835 5913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.438067 5896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:08.438191 5896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "e92966c15b754324a91025b430e79b38"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3b8127eefb6774661ae3f2b990693fa6"
server_key_iv: "5c8b7096cba1f63ca7285247fa395334"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.438275 5896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:08.465122 5896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:08.467649 5896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:08.467836 5896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:08.468123 5896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:08.468468 5896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:08.468526 5896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.468573 5896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:08.468607 5896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.475195 5896 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:38435
I20260808 22:55:08.475243 6039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:38435 every 8 connection(s)
I20260808 22:55:08.475610 5896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:08.485918 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 5896
I20260808 22:55:08.486060 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:08.486280 1872 external_mini_cluster.cc:1428] Setting key 11ab0dc4d14d5e4c30c9d893ba43158c
I20260808 22:55:08.487154 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.476418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:53559 (local address 127.1.212.62:44493)
0808 22:55:08.476468 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.476475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.476752 (+ 277us) server_negotiation.cc:407] Connection header received
0808 22:55:08.477939 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.477940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.477958 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.478027 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.478788 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.479303 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.480461 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.480655 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.480710 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.483577 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.483578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.483579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.483588 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.485168 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.485673 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.485675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.485675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.485740 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.486233 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.486235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.486235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.486435 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.486540 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.486946 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.487047 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:55:08.487406 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.476384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:44493 (local address 127.1.212.2:53559)
0808 22:55:08.476586 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:08.476654 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:08.477820 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:08.478162 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:08.478168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:08.478474 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:08.478694 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.478698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.479435 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.479436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.480371 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.480374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.480743 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.480744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.480923 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.481572 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:08.481585 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:08.483465 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:08.485273 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.485284 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.485287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.485578 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.485819 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.485821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.485821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.486064 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.486537 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:08.486546 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:08.486762 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 22:55:08.486912 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 22:55:08.487122 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260808 22:55:08.487934 6040 heartbeater.cc:347] Connected to a master server at 127.1.212.62:44493
I20260808 22:55:08.488085 6040 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:08.488314 6040 heartbeater.cc:511] Master 127.1.212.62:44493 requested a full tablet report, sending...
I20260808 22:55:08.488765 5679 ts_manager.cc:194] Registered new tserver with Master: e92966c15b754324a91025b430e79b38 (127.1.212.2:38435)
I20260808 22:55:08.489470 5679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:53559
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:08.516048 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:08.605638 6046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:08.605893 6046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:08.605923 6046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:08.607120 6046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:08.607160 6046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:08.607187 6046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:08.608832 6046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:44493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:08.609107 6046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:08.609309 6046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:08.614981 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.615115 6046 server_base.cc:1034] running on GCE node
W20260808 22:55:08.614984 6051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:08.614981 6052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:08.615615 6046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:08.615935 6046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:08.617113 6046 hybrid_clock.cc:648] HybridClock initialized: now 1786229708617106 us; error 37 us; skew 500 ppm
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:08.619760 6046 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:08.620281 6046 webserver.cc:460] Webserver started at http://127.1.212.3:39033/ using document root <none> and password file <none>
I20260808 22:55:08.620538 6046 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:08.620612 6046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:08.620890 6046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:08.622058 6046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "64e1365ba8244012953c7b82bbeef0a0"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3abe32aa21219d867e0cf40b5a9048a3"
server_key_iv: "d8d2542d8a7680bd8830d096f05c8e15"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.622460 6046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "64e1365ba8244012953c7b82bbeef0a0"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3abe32aa21219d867e0cf40b5a9048a3"
server_key_iv: "d8d2542d8a7680bd8830d096f05c8e15"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.623921 6046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:08.624920 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.625121 6046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:08.625236 6046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "64e1365ba8244012953c7b82bbeef0a0"
format_stamp: "Formatted at 2026-08-08 22:55:08 on dist-test-slave-1x4g"
server_key: "3abe32aa21219d867e0cf40b5a9048a3"
server_key_iv: "d8d2542d8a7680bd8830d096f05c8e15"
server_key_version: "encryptionkey@0"
I20260808 22:55:08.625327 6046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:08.649147 6046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:08.651429 6046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:08.651628 6046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:08.651897 6046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:08.652266 6046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:08.652331 6046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.652387 6046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:08.652418 6046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:08.659144 6046 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:39385
I20260808 22:55:08.659194 6181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:39385 every 8 connection(s)
I20260808 22:55:08.659574 6046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:08.662734 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6046
I20260808 22:55:08.662858 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:08.663045 1872 external_mini_cluster.cc:1428] Setting key 109418800b0bb7ac5426de2170ba6289
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:08.669766 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.660373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:43279 (local address 127.1.212.62:44493)
0808 22:55:08.660471 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.660474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.660677 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:55:08.661735 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.661737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.661750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.661808 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.662482 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.663079 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.664212 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.664444 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.664507 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.666908 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.666910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.666910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.666918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.668422 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.668876 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.668877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.668877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.668937 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.669176 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.669176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.669177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.669291 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.669341 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.669624 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.669695 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:55:08.669936 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.660326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:44493 (local address 127.1.212.3:43279)
0808 22:55:08.660526 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:08.660570 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:08.661619 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:08.661873 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:08.661877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:08.662178 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:08.662394 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.662399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.663230 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.663231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.664121 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:08.664124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.664547 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.664548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:08.664689 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.665279 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:08.665288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:08.666801 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:08.668507 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.668514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.668516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.668742 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.669001 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:08.669001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:08.669002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:08.669114 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:08.669355 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:08.669361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:08.669502 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 22:55:08.669584 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:55:08.669720 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260808 22:55:08.670499 6182 heartbeater.cc:347] Connected to a master server at 127.1.212.62:44493
I20260808 22:55:08.670629 6182 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:08.670832 6182 heartbeater.cc:511] Master 127.1.212.62:44493 requested a full tablet report, sending...
I20260808 22:55:08.671314 5679 ts_manager.cc:194] Registered new tserver with Master: 64e1365ba8244012953c7b82bbeef0a0 (127.1.212.3:39385)
I20260808 22:55:08.671720 5679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:43279
I20260808 22:55:08.674787 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:55:08.701350 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.692848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.1.212.1:41421)
0808 22:55:08.693022 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.693027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.693038 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:08.693066 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.693068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.693083 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.693169 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.693826 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.694567 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.695369 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.695728 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.695825 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.698420 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.698433 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.698433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.698448 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.699955 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.700344 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.700348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.700349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.700425 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.700647 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.700648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.700648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.700880 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.701075 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.701124 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.701218 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[5630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:08.710270 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:08.702874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.1.212.62:44493)
0808 22:55:08.702949 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:08.702951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:08.702960 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:08.703026 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:08.703027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:08.703038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:08.703088 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:08.703584 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.704105 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.704912 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:08.705105 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:08.705189 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:08.707581 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:08.707583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:08.707583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:08.707591 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:08.709089 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.709446 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.709447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.709447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.709502 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:08.709684 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:08.709685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:08.709685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:08.709843 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:08.709925 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:08.710119 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 22:55:08.710200 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:55:08.710585 5679 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60168
I20260808 22:55:08.711030 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5747
I20260808 22:55:08.720477 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5896
I20260808 22:55:08.728919 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6046
I20260808 22:55:08.737223 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 5646
2026-08-08T22:55:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1976 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[6197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[6197](info): set up 2 sockets
Aug 08 22:55:08 dist-test-slave-1x4g krb5kdc[6197](info): commencing operation
krb5kdc: starting...
W20260808 22:55:09.828977 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:09 dist-test-slave-1x4g krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:09.964151 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39659
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:39659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:10.051437 6212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:10.051715 6212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:10.051757 6212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:10.053030 6212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:10.053107 6212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:10.053130 6212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:10.053161 6212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:10.053192 6212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:10.054800 6212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:39659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:10.055124 6212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:10.055382 6212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:10.060815 6218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.060762 6217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.060801 6212 server_base.cc:1034] running on GCE node
W20260808 22:55:10.060977 6220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.061446 6212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:10.061793 6212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:10.062988 6212 hybrid_clock.cc:648] HybridClock initialized: now 1786229710062975 us; error 47 us; skew 500 ppm
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:10.069298 6212 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:10.069715 6212 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:10.069794 6212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:10.069967 6212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:10.071179 6212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f08edc9607bb465bb3826898060d927c"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "53a7a2517b5f178297eb5320cc5faa17"
server_key_iv: "e7dab26072a8631fa604500d530ca28b"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.071616 6212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f08edc9607bb465bb3826898060d927c"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "53a7a2517b5f178297eb5320cc5faa17"
server_key_iv: "e7dab26072a8631fa604500d530ca28b"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.073143 6212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 22:55:10.073987 6226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.074187 6212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:10.074311 6212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f08edc9607bb465bb3826898060d927c"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "53a7a2517b5f178297eb5320cc5faa17"
server_key_iv: "e7dab26072a8631fa604500d530ca28b"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.074396 6212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:10.096550 6212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:10.099059 6212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:10.099267 6212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:10.103621 6212 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:39659
I20260808 22:55:10.103667 6280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:39659 every 8 connection(s)
I20260808 22:55:10.103995 6212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:10.105145 6281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:10.107654 6281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: Bootstrap starting.
I20260808 22:55:10.108340 6281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:10.108671 6281 log.cc:824] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:10.109489 6281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: No bootstrap required, opened a new log
I20260808 22:55:10.111171 6281 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } }
I20260808 22:55:10.111323 6281 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:10.111369 6281 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08edc9607bb465bb3826898060d927c, State: Initialized, Role: FOLLOWER
I20260808 22:55:10.111531 6281 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } }
I20260808 22:55:10.111615 6281 raft_consensus.cc:388] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:10.111608 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6212
I20260808 22:55:10.111676 6281 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:10.111720 6281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:10.111770 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:10.111971 1872 external_mini_cluster.cc:1428] Setting key 798d887b51753da8bdc1790ae675803d
I20260808 22:55:10.112907 6281 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } }
I20260808 22:55:10.113009 6281 leader_election.cc:302] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f08edc9607bb465bb3826898060d927c; no voters:
I20260808 22:55:10.113219 6281 leader_election.cc:288] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:10.113253 6286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:10.113452 6286 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [term 1 LEADER]: Becoming Leader. State: Replica: f08edc9607bb465bb3826898060d927c, State: Running, Role: LEADER
I20260808 22:55:10.113546 6286 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } }
I20260808 22:55:10.113592 6281 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:10.113873 6286 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08edc9607bb465bb3826898060d927c. Latest consensus state: current_term: 1 leader_uuid: "f08edc9607bb465bb3826898060d927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } } }
I20260808 22:55:10.113961 6286 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:10.113888 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f08edc9607bb465bb3826898060d927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08edc9607bb465bb3826898060d927c" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39659 } } }
I20260808 22:55:10.114152 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:10.115003 6291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:10.115540 6291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:10.117161 6291 catalog_manager.cc:1347] Generated new cluster ID: 01713edfcaa245dd817fd94d65165663
I20260808 22:55:10.117229 6291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:10.124599 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.112557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.1.212.62:39659)
0808 22:55:10.112795 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.112806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.112843 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:55:10.112958 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.112964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.113042 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.113200 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.114025 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.115475 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.116536 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.117199 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.117314 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.120572 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.120579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.120579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:10.120595 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.122722 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.123180 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.123183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.123184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.123308 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.123549 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.123550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.123550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.123880 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.124025 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.124170 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.124322 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260808 22:55:10.151268 6291 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:10.151906 6291 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:10.156052 6291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: Generated new TSK 0
I20260808 22:55:10.156280 6291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:10.196676 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:10.287067 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:10.287359 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:10.287407 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:10.288724 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:10.288865 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:10.288954 6309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:10.291059 6309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:10.291437 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:10.291723 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:10.297152 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.297140 6315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.297140 6317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.297427 6309 server_base.cc:1034] running on GCE node
I20260808 22:55:10.297708 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:10.297983 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:10.299168 6309 hybrid_clock.cc:648] HybridClock initialized: now 1786229710299154 us; error 52 us; skew 500 ppm
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:10.302081 6309 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:10.302486 6309 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:10.302543 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:10.302685 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:10.303859 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "f077812126694e4cb96be6915ec3fa57"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "36e9727d54fe9447aceaf23b73e55d6a"
server_key_iv: "009db09c2678eca12d7c4ccef844879c"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.304257 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "f077812126694e4cb96be6915ec3fa57"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "36e9727d54fe9447aceaf23b73e55d6a"
server_key_iv: "009db09c2678eca12d7c4ccef844879c"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.305881 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:10.306768 6323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.307085 6309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:10.307178 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "f077812126694e4cb96be6915ec3fa57"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "36e9727d54fe9447aceaf23b73e55d6a"
server_key_iv: "009db09c2678eca12d7c4ccef844879c"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.307242 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:10.317092 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:10.319322 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:10.319468 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:10.319670 6309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:10.320020 6309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:10.320057 6309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.320081 6309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:10.320093 6309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.326319 6309 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:46413
I20260808 22:55:10.326404 6444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:46413 every 8 connection(s)
I20260808 22:55:10.326745 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:10.333485 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6309
I20260808 22:55:10.333626 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:10.333825 1872 external_mini_cluster.cc:1428] Setting key 1cc358577ed4be6d86c0d81159cf7740
I20260808 22:55:10.337814 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.327582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51429 (local address 127.1.212.62:39659)
0808 22:55:10.327638 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.327639 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.327871 (+ 232us) server_negotiation.cc:407] Connection header received
0808 22:55:10.328374 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.328375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.328385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.328459 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.329512 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.330024 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.331140 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.331354 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.331426 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.333961 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.333964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.333964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:10.333980 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.336082 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.336621 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.336623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.336623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.336698 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.337070 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.337070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.337071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.337196 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.337254 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.337656 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.337745 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:55:10.338120 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.327539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39659 (local address 127.1.212.1:51429)
0808 22:55:10.327723 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.327768 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.328281 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.328559 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.328563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.328873 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:10.329412 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.329418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.330157 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.330158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.331030 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.331032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.331443 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.331444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.331551 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.332175 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:10.332185 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:10.333835 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:10.336221 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.336232 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.336235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.336519 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.336799 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.336800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.336800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.336989 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.337266 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:10.337274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:10.337493 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.337614 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.337833 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260808 22:55:10.338778 6445 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39659
I20260808 22:55:10.338881 6445 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:10.339078 6445 heartbeater.cc:511] Master 127.1.212.62:39659 requested a full tablet report, sending...
I20260808 22:55:10.339679 6243 ts_manager.cc:194] Registered new tserver with Master: f077812126694e4cb96be6915ec3fa57 (127.1.212.1:46413)
I20260808 22:55:10.340378 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:51429
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:10.364924 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:10.453186 6450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:10.453501 6450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:10.453549 6450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:10.454949 6450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:10.455052 6450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:10.455106 6450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:10.457079 6450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:10.457432 6450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:10.457671 6450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:10.463346 6456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.463430 6459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.463444 6457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.463635 6450 server_base.cc:1034] running on GCE node
I20260808 22:55:10.463846 6450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:10.464100 6450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:10.465282 6450 hybrid_clock.cc:648] HybridClock initialized: now 1786229710465227 us; error 76 us; skew 500 ppm
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:10.468189 6450 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:10.468609 6450 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:10.468664 6450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:10.468767 6450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:10.470100 6450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "951616ed46974c15b4a47e16df496fd1"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "2c7eb8cb36e71d34e4b07316ca9612b0"
server_key_iv: "a0582fa8edbec37db415557df38a0a76"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.470460 6450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "951616ed46974c15b4a47e16df496fd1"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "2c7eb8cb36e71d34e4b07316ca9612b0"
server_key_iv: "a0582fa8edbec37db415557df38a0a76"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.471937 6450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:10.472878 6466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.473117 6450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:10.473304 6450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "951616ed46974c15b4a47e16df496fd1"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "2c7eb8cb36e71d34e4b07316ca9612b0"
server_key_iv: "a0582fa8edbec37db415557df38a0a76"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.473402 6450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:10.488101 6450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:10.491071 6450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:10.491201 6450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:10.491468 6450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:10.491787 6450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:10.491822 6450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.491844 6450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:10.491856 6450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.498507 6450 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:33167
I20260808 22:55:10.498533 6593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:33167 every 8 connection(s)
I20260808 22:55:10.498878 6450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:10.501597 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6450
I20260808 22:55:10.501700 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:10.501904 1872 external_mini_cluster.cc:1428] Setting key 065492e11ccd371ece9a593ce0bc389a
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:10.510562 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.499701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:52377 (local address 127.1.212.62:39659)
0808 22:55:10.499799 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.499802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.500011 (+ 209us) server_negotiation.cc:407] Connection header received
0808 22:55:10.500552 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.500553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.500565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.500612 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.501845 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.502524 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.503705 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.503922 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.504021 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.506952 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.506955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.506955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:10.506966 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.508670 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.509328 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.509330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.509330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.509388 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.509782 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.509783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.509783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.509906 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.509959 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.510391 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.510492 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:55:10.510869 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.499660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39659 (local address 127.1.212.2:52377)
0808 22:55:10.499869 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.499915 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.500436 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.500706 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.500711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.501089 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:10.501723 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.501731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.502744 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.502745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.503599 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.503602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.504035 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.504036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.504174 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.504830 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:10.504842 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:10.506838 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:10.508804 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.508817 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.508820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.509239 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.509465 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.509466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.509466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.509643 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.509979 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:10.509988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:10.510202 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.510325 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.510594 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260808 22:55:10.511515 6594 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39659
I20260808 22:55:10.511641 6594 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:10.511831 6594 heartbeater.cc:511] Master 127.1.212.62:39659 requested a full tablet report, sending...
I20260808 22:55:10.512243 6243 ts_manager.cc:194] Registered new tserver with Master: 951616ed46974c15b4a47e16df496fd1 (127.1.212.2:33167)
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
I20260808 22:55:10.512656 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:52377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:10.533102 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:10.622413 6600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:10.622651 6600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:10.622678 6600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:10.624578 6600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:10.624671 6600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:10.624728 6600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:10.626593 6600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:10.626893 6600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:10.627117 6600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:10.633373 6606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.633373 6605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.633533 6608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.633560 6600 server_base.cc:1034] running on GCE node
I20260808 22:55:10.634034 6600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:10.634334 6600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:10.635542 6600 hybrid_clock.cc:648] HybridClock initialized: now 1786229710635517 us; error 43 us; skew 500 ppm
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:10.638641 6600 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:10.639114 6600 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:10.639204 6600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:10.639369 6600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:10.640615 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "3eed0f18dfb1a45777a6a867a813143e"
server_key_iv: "ba992384b1b7db736900973fb69b1500"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.641089 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "3eed0f18dfb1a45777a6a867a813143e"
server_key_iv: "ba992384b1b7db736900973fb69b1500"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.642529 6600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:10.643970 6614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.644227 6600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:10.644371 6600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "3eed0f18dfb1a45777a6a867a813143e"
server_key_iv: "ba992384b1b7db736900973fb69b1500"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.644480 6600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:10.658710 6600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:10.661027 6600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:10.661216 6600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:10.661476 6600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:10.661818 6600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:10.661875 6600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.661921 6600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:10.661953 6600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.668164 6600 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:35559
I20260808 22:55:10.668207 6743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:35559 every 8 connection(s)
I20260808 22:55:10.668540 6600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:10.669873 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6600
I20260808 22:55:10.669983 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:10.670176 1872 external_mini_cluster.cc:1428] Setting key 14c72532f59b8e7d5d8c824d82393e14
Aug 08 22:55:10 dist-test-slave-1x4g krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:10.678908 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.669412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34023 (local address 127.1.212.62:39659)
0808 22:55:10.669489 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.669491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.669677 (+ 186us) server_negotiation.cc:407] Connection header received
0808 22:55:10.670451 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.670452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.670467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.670527 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.671737 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.672237 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.673359 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.673552 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.673615 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.675875 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.675878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.675878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:10.675888 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.677386 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.677859 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.677861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.677863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.677923 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.678203 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.678204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.678204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.678332 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.678409 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.678726 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.678812 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:55:10.679126 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.669361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39659 (local address 127.1.212.3:34023)
0808 22:55:10.669534 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.669581 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.670324 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.670607 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.670628 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.671037 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:10.671637 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.671643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.672328 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.672329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.673274 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.673277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.673649 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.673650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.673763 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.674289 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:10.674297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:10.675763 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:10.677482 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.677492 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.677495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.677765 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.677998 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:10.677999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:10.678000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:10.678124 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:10.678412 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:10.678420 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:10.678580 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.678691 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.678870 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260808 22:55:10.679693 6744 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39659
I20260808 22:55:10.679808 6744 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:10.679993 6744 heartbeater.cc:511] Master 127.1.212.62:39659 requested a full tablet report, sending...
I20260808 22:55:10.680444 6243 ts_manager.cc:194] Registered new tserver with Master: f0d41af8d77a49a7b02e0e6ca2cd4880 (127.1.212.3:35559)
I20260808 22:55:10.680853 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:34023
I20260808 22:55:10.681985 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:55:10.689963 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.683355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.1.212.62:39659)
0808 22:55:10.683444 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.683446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.683544 (+ 98us) server_negotiation.cc:407] Connection header received
0808 22:55:10.683624 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.683625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.683637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.683695 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.684295 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.684848 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.685634 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.685841 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.685922 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.687239 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.687241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.687241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:10.687249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.688589 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.689034 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.689036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.689036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.689106 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:10.689326 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:10.689327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:10.689328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:10.689503 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.689623 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.689759 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.689855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:55:10.691948 6243 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:10.694162 6243 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:10.701956 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.696928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46413 (local address 127.0.0.1:59480)
0808 22:55:10.697341 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.697350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.697953 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.698447 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.698448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.698450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:10.698500 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.698502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699377 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.699379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.700602 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.700605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701588 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.701684 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701701 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.701781 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.701846 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":78,"thread_start_us":77,"threads_started":1}
I20260808 22:55:10.702117 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.696926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:35559 (local address 127.0.0.1:55762)
0808 22:55:10.697126 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.697175 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.697253 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.697834 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.697837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.697841 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:10.697935 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.697937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699448 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.699449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.701009 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.701011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701808 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.701899 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701904 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.701955 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.702006 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":148,"thread_start_us":133,"threads_started":1}
I20260808 22:55:10.702304 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.697049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.1.212.3:35559)
0808 22:55:10.697633 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.697638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.697649 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:10.697696 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.697698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.697715 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.697817 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:10.698101 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699340 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701113 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701709 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701790 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701848 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.702174 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.702222 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":79,"threads_started":1}
I20260808 22:55:10.702364 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.697144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.1.212.1:46413)
0808 22:55:10.697337 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.697343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.697523 (+ 180us) server_negotiation.cc:407] Connection header received
0808 22:55:10.698307 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.698311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.698334 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.698447 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:10.698564 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699271 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.700705 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701467 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701558 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701625 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.702215 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.702271 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260808 22:55:10.702620 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.697552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.1.212.2:33167)
0808 22:55:10.697781 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.697785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.697795 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:10.697837 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.697840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.697855 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.697927 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:10.698171 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699340 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701094 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701838 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701919 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701983 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.702446 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.702502 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":113,"threads_started":1}
I20260808 22:55:10.702833 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.697189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:33167 (local address 127.0.0.1:54656)
0808 22:55:10.697400 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.697411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.697670 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.698015 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.698016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.698018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:10.698081 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.698083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.699605 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.699606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.700988 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.700992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.701920 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.701921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.701989 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.701994 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.702052 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.702103 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":110,"thread_start_us":72,"threads_started":1}
I20260808 22:55:10.703188 6668 tablet_service.cc:1467] Processing CreateTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf (DEFAULT_TABLE table=test-table [id=8e9b936a78194f9c8a830512d1a42728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:10.703328 6376 tablet_service.cc:1467] Processing CreateTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf (DEFAULT_TABLE table=test-table [id=8e9b936a78194f9c8a830512d1a42728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:10.703534 6518 tablet_service.cc:1467] Processing CreateTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf (DEFAULT_TABLE table=test-table [id=8e9b936a78194f9c8a830512d1a42728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:10.703624 6376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2e9e98a022e49368d58b2c9ec2cefaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:10.703624 6668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2e9e98a022e49368d58b2c9ec2cefaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:10.703882 6518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2e9e98a022e49368d58b2c9ec2cefaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:10.707053 6764 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Bootstrap starting.
I20260808 22:55:10.707660 6767 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: Bootstrap starting.
I20260808 22:55:10.707902 6764 tablet_bootstrap.cc:654] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:10.708348 6767 tablet_bootstrap.cc:654] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:10.708712 6764 log.cc:824] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:10.709004 6766 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Bootstrap starting.
I20260808 22:55:10.709075 6767 log.cc:824] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:10.709656 6766 tablet_bootstrap.cc:654] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:10.709666 6764 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: No bootstrap required, opened a new log
I20260808 22:55:10.709756 6764 ts_tablet_manager.cc:1397] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 22:55:10.710268 6766 log.cc:824] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:10.711126 6766 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: No bootstrap required, opened a new log
I20260808 22:55:10.711189 6766 ts_tablet_manager.cc:1397] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:55:10.711512 6764 raft_consensus.cc:348] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.711685 6764 raft_consensus.cc:374] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:10.711752 6764 raft_consensus.cc:729] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 951616ed46974c15b4a47e16df496fd1, State: Initialized, Role: FOLLOWER
I20260808 22:55:10.711896 6764 consensus_queue.cc:260] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.712248 6764 ts_tablet_manager.cc:1428] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 22:55:10.712261 6594 heartbeater.cc:503] Master 127.1.212.62:39659 was elected leader, sending a full tablet report...
I20260808 22:55:10.712992 6767 tablet_bootstrap.cc:492] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: No bootstrap required, opened a new log
I20260808 22:55:10.713078 6767 ts_tablet_manager.cc:1397] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:55:10.713996 6766 raft_consensus.cc:348] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.714144 6766 raft_consensus.cc:374] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:10.714177 6766 raft_consensus.cc:729] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d41af8d77a49a7b02e0e6ca2cd4880, State: Initialized, Role: FOLLOWER
I20260808 22:55:10.714347 6766 consensus_queue.cc:260] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.714704 6766 ts_tablet_manager.cc:1428] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:10.714756 6744 heartbeater.cc:503] Master 127.1.212.62:39659 was elected leader, sending a full tablet report...
I20260808 22:55:10.715525 6767 raft_consensus.cc:348] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.715762 6767 raft_consensus.cc:374] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:10.715816 6767 raft_consensus.cc:729] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f077812126694e4cb96be6915ec3fa57, State: Initialized, Role: FOLLOWER
I20260808 22:55:10.715950 6767 consensus_queue.cc:260] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.716298 6767 ts_tablet_manager.cc:1428] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:10.716315 6445 heartbeater.cc:503] Master 127.1.212.62:39659 was elected leader, sending a full tablet report...
W20260808 22:55:10.749477 6595 tablet.cc:2369] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:10.753208 6772 raft_consensus.cc:482] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:10.753396 6772 raft_consensus.cc:504] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.753821 6772 leader_election.cc:288] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f077812126694e4cb96be6915ec3fa57 (127.1.212.1:46413), 951616ed46974c15b4a47e16df496fd1 (127.1.212.2:33167)
I20260808 22:55:10.757215 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.754208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34619 (local address 127.1.212.2:33167)
0808 22:55:10.754280 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.754282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.754295 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:10.754330 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.754331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.754351 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.754457 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:10.754664 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.755218 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756287 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.756733 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756876 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.756889 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.757075 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.757120 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:55:10.757227 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.754268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:33241 (local address 127.1.212.1:46413)
0808 22:55:10.754355 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.754357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.754586 (+ 229us) server_negotiation.cc:407] Connection header received
0808 22:55:10.754622 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.754624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.754639 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.754701 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:10.754839 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.755334 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756342 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.756746 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756893 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.756906 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.757107 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.757151 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:55:10.757249 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.754062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:33167 (local address 127.1.212.3:34619)
0808 22:55:10.754120 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.754133 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.754183 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.754475 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.754481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.754483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:10.754550 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.754552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.755320 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.755322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.756178 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.756181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.756889 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.756990 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.756994 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.757066 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.757120 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 22:55:10.757263 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.754077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46413 (local address 127.1.212.3:33241)
0808 22:55:10.754496 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.754508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:10.754574 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:10.754697 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:10.754698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:10.754699 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:10.754771 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.754773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.755429 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.755429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.756215 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:10.756218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.756905 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.756906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:10.757016 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.757020 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:10.757098 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:55:10.757143 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":158,"thread_start_us":173,"threads_started":1}
I20260808 22:55:10.757766 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2e9e98a022e49368d58b2c9ec2cefaf" candidate_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951616ed46974c15b4a47e16df496fd1" is_pre_election: true
I20260808 22:55:10.757812 6396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2e9e98a022e49368d58b2c9ec2cefaf" candidate_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f077812126694e4cb96be6915ec3fa57" is_pre_election: true
I20260808 22:55:10.757961 6542 raft_consensus.cc:2454] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d41af8d77a49a7b02e0e6ca2cd4880 in term 0.
I20260808 22:55:10.757968 6396 raft_consensus.cc:2454] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d41af8d77a49a7b02e0e6ca2cd4880 in term 0.
I20260808 22:55:10.758430 6617 leader_election.cc:302] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 951616ed46974c15b4a47e16df496fd1, f077812126694e4cb96be6915ec3fa57, f0d41af8d77a49a7b02e0e6ca2cd4880; no voters:
I20260808 22:55:10.758639 6772 raft_consensus.cc:2781] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:10.758723 6772 raft_consensus.cc:482] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:10.758759 6772 raft_consensus.cc:3037] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:10.759552 6772 raft_consensus.cc:504] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.759841 6772 leader_election.cc:288] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [CANDIDATE]: Term 1 election: Requested vote from peers f077812126694e4cb96be6915ec3fa57 (127.1.212.1:46413), 951616ed46974c15b4a47e16df496fd1 (127.1.212.2:33167)
I20260808 22:55:10.759948 6396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2e9e98a022e49368d58b2c9ec2cefaf" candidate_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f077812126694e4cb96be6915ec3fa57"
I20260808 22:55:10.760023 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2e9e98a022e49368d58b2c9ec2cefaf" candidate_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951616ed46974c15b4a47e16df496fd1"
I20260808 22:55:10.760118 6542 raft_consensus.cc:3037] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:10.760066 6396 raft_consensus.cc:3037] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:10.761019 6542 raft_consensus.cc:2454] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d41af8d77a49a7b02e0e6ca2cd4880 in term 1.
I20260808 22:55:10.761138 6396 raft_consensus.cc:2454] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d41af8d77a49a7b02e0e6ca2cd4880 in term 1.
I20260808 22:55:10.761241 6616 leader_election.cc:302] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 951616ed46974c15b4a47e16df496fd1, f0d41af8d77a49a7b02e0e6ca2cd4880; no voters:
I20260808 22:55:10.761349 6772 raft_consensus.cc:2781] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:10.761534 6772 raft_consensus.cc:686] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 1 LEADER]: Becoming Leader. State: Replica: f0d41af8d77a49a7b02e0e6ca2cd4880, State: Running, Role: LEADER
I20260808 22:55:10.761658 6772 consensus_queue.cc:237] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } }
I20260808 22:55:10.762667 6243 catalog_manager.cc:5495] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d41af8d77a49a7b02e0e6ca2cd4880 (127.1.212.3). New cstate: current_term: 1 leader_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d41af8d77a49a7b02e0e6ca2cd4880" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 } health_report { overall_health: UNKNOWN } } }
I20260808 22:55:10.780534 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.777931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.1.212.3:35559)
0808 22:55:10.778023 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.778025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.778034 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:10.778082 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.778083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.778106 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.778193 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:10.778365 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.778861 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.779652 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:10.779875 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:10.779956 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:10.780147 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:10.780272 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:10.780373 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.780393 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.780416 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:55:10.783056 6542 raft_consensus.cc:1261] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 1 FOLLOWER]: Refusing update from remote peer f0d41af8d77a49a7b02e0e6ca2cd4880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:10.783074 6396 raft_consensus.cc:1261] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 1 FOLLOWER]: Refusing update from remote peer f0d41af8d77a49a7b02e0e6ca2cd4880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:10.783474 6772 consensus_queue.cc:1035] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "951616ed46974c15b4a47e16df496fd1" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 33167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:10.783649 6775 consensus_queue.cc:1035] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f077812126694e4cb96be6915ec3fa57" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:10.786396 6779 mvcc.cc:205] Tried to move back new op lower bound from 7316396895364222976 to 7316396895280586752. Current Snapshot: MvccSnapshot[applied={T|T < 7316396895364222976}]
I20260808 22:55:10.786589 6781 mvcc.cc:205] Tried to move back new op lower bound from 7316396895364222976 to 7316396895280586752. Current Snapshot: MvccSnapshot[applied={T|T < 7316396895364222976}]
I20260808 22:55:10.787106 6780 mvcc.cc:205] Tried to move back new op lower bound from 7316396895364222976 to 7316396895280586752. Current Snapshot: MvccSnapshot[applied={T|T < 7316396895364222976}]
I20260808 22:55:10.789155 6243 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:10.789283 6243 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:10.790282 6243 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f08edc9607bb465bb3826898060d927c: Sending DeleteTablet for 3 replicas of tablet f2e9e98a022e49368d58b2c9ec2cefaf
I20260808 22:55:10.790702 6376 tablet_service.cc:1514] Processing DeleteTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:10 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:59480
I20260808 22:55:10.790746 6668 tablet_service.cc:1514] Processing DeleteTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:10 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:55762
I20260808 22:55:10.790985 6790 tablet_replica.cc:330] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: stopping tablet replica
I20260808 22:55:10.791014 6518 tablet_service.cc:1514] Processing DeleteTablet for tablet f2e9e98a022e49368d58b2c9ec2cefaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:10 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:54656
I20260808 22:55:10.791133 6790 raft_consensus.cc:2229] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:10.791257 6793 tablet_replica.cc:330] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: stopping tablet replica
I20260808 22:55:10.791290 6790 raft_consensus.cc:2258] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:10.791366 6793 raft_consensus.cc:2229] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:10.791487 6793 raft_consensus.cc:2258] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:10.791546 6792 tablet_replica.cc:330] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57: stopping tablet replica
I20260808 22:55:10.791667 6792 raft_consensus.cc:2229] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:10.791798 6792 raft_consensus.cc:2258] T f2e9e98a022e49368d58b2c9ec2cefaf P f077812126694e4cb96be6915ec3fa57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:10.791966 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6309
I20260808 22:55:10.792037 6793 ts_tablet_manager.cc:1905] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:10.794767 6793 ts_tablet_manager.cc:1918] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:10.794895 6793 log.cc:1196] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/f2e9e98a022e49368d58b2c9ec2cefaf
I20260808 22:55:10.795290 6793 ts_tablet_manager.cc:1939] T f2e9e98a022e49368d58b2c9ec2cefaf P 951616ed46974c15b4a47e16df496fd1: Deleting consensus metadata
I20260808 22:55:10.796048 6228 catalog_manager.cc:4841] TS 951616ed46974c15b4a47e16df496fd1 (127.1.212.2:33167): tablet f2e9e98a022e49368d58b2c9ec2cefaf (table test-table [id=8e9b936a78194f9c8a830512d1a42728]) successfully deleted
I20260808 22:55:10.801112 6790 ts_tablet_manager.cc:1905] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:10.803323 6790 ts_tablet_manager.cc:1918] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:10.803448 6790 log.cc:1196] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/f2e9e98a022e49368d58b2c9ec2cefaf
I20260808 22:55:10.803758 6790 ts_tablet_manager.cc:1939] T f2e9e98a022e49368d58b2c9ec2cefaf P f0d41af8d77a49a7b02e0e6ca2cd4880: Deleting consensus metadata
I20260808 22:55:10.804302 6228 catalog_manager.cc:4841] TS f0d41af8d77a49a7b02e0e6ca2cd4880 (127.1.212.3:35559): tablet f2e9e98a022e49368d58b2c9ec2cefaf (table test-table [id=8e9b936a78194f9c8a830512d1a42728]) successfully deleted
W20260808 22:55:10.807858 6229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:46413 (error 108)
I20260808 22:55:10.808008 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6450
I20260808 22:55:10.809239 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.809058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46413 (local address 127.0.0.1:59492)
0808 22:55:10.809134 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:10.809187 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 22:55:10.809969 6229 catalog_manager.cc:4584] TS f077812126694e4cb96be6915ec3fa57 (127.1.212.1:46413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2e9e98a022e49368d58b2c9ec2cefaf: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46413: connect: Connection refused (error 111)
I20260808 22:55:10.815986 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6600
I20260808 22:55:10.824534 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6212
2026-08-08T22:55:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2091 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T22:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:10Z Disabled control of system clock
I20260808 22:55:10.862536 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40827
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:55:10.950299 6800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:10.950544 6800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:10.950568 6800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:10.951749 6800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:10.951789 6800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:10.951802 6800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:10.951815 6800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:10.951828 6800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:10.953311 6800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:10.953567 6800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:10.953840 6800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:10.959125 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.959255 6800 server_base.cc:1034] running on GCE node
W20260808 22:55:10.959383 6805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:10.959125 6809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:10.959686 6800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:10.960021 6800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:10.961193 6800 hybrid_clock.cc:648] HybridClock initialized: now 1786229710961168 us; error 38 us; skew 500 ppm
I20260808 22:55:10.962595 6800 webserver.cc:460] Webserver started at http://127.1.212.62:33807/ using document root <none> and password file <none>
I20260808 22:55:10.962821 6800 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:10.962865 6800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:10.962967 6800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:10.964270 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "abb743ce7dec4588b10cd84c9a171be4"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "896710bedb001646fc07d22be4c46a9c"
server_key_iv: "a3a5625d6b4c9a7134396433f3e07443"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.964627 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "abb743ce7dec4588b10cd84c9a171be4"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "896710bedb001646fc07d22be4c46a9c"
server_key_iv: "a3a5625d6b4c9a7134396433f3e07443"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.966207 6800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:10.967250 6815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:10.967475 6800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:10.967608 6800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "abb743ce7dec4588b10cd84c9a171be4"
format_stamp: "Formatted at 2026-08-08 22:55:10 on dist-test-slave-1x4g"
server_key: "896710bedb001646fc07d22be4c46a9c"
server_key_iv: "a3a5625d6b4c9a7134396433f3e07443"
server_key_version: "encryptionkey@0"
I20260808 22:55:10.967741 6800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:10.972479 6800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:10.972816 6800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:10.972965 6800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:10.977185 6800 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:40827
I20260808 22:55:10.977295 6874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:40827 every 8 connection(s)
I20260808 22:55:10.977555 6800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:10.978343 6876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:10.978399 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6800
I20260808 22:55:10.978504 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:10.978694 1872 external_mini_cluster.cc:1428] Setting key a34d3a94f12a3c6cd62df801ceee40b6
I20260808 22:55:10.982197 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:10.979696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.1.212.62:40827)
0808 22:55:10.980000 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:10.980012 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:10.980055 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:55:10.980536 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:10.980546 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:10.980630 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:10.980850 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:10.981675 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:10.981679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:10.981679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:10.981685 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:10.981742 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:10.981757 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:10.981840 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:10.981895 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:55:10.981958 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":125,"threads_started":1}
I20260808 22:55:10.983012 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: Bootstrap starting.
I20260808 22:55:10.983680 6876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:10.983978 6876 log.cc:824] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:10.984830 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: No bootstrap required, opened a new log
I20260808 22:55:10.986335 6876 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } }
I20260808 22:55:10.986527 6876 raft_consensus.cc:374] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:10.986581 6876 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb743ce7dec4588b10cd84c9a171be4, State: Initialized, Role: FOLLOWER
I20260808 22:55:10.986724 6876 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } }
I20260808 22:55:10.986840 6876 raft_consensus.cc:388] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:10.986886 6876 raft_consensus.cc:482] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:10.986939 6876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:10.987537 6876 raft_consensus.cc:504] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } }
I20260808 22:55:10.987668 6876 leader_election.cc:302] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abb743ce7dec4588b10cd84c9a171be4; no voters:
I20260808 22:55:10.987875 6876 leader_election.cc:288] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:10.988181 6876 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:10.988276 6883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:10.988432 6883 raft_consensus.cc:686] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [term 1 LEADER]: Becoming Leader. State: Replica: abb743ce7dec4588b10cd84c9a171be4, State: Running, Role: LEADER
I20260808 22:55:10.988550 6883 consensus_queue.cc:237] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } }
I20260808 22:55:10.989027 6884 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abb743ce7dec4588b10cd84c9a171be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } } }
I20260808 22:55:10.989058 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb743ce7dec4588b10cd84c9a171be4. Latest consensus state: current_term: 1 leader_uuid: "abb743ce7dec4588b10cd84c9a171be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb743ce7dec4588b10cd84c9a171be4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40827 } } }
I20260808 22:55:10.989130 6884 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:10.989136 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:10.989351 6892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:10.990056 6892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:10.991590 6892 catalog_manager.cc:1347] Generated new cluster ID: 53e41efcea53489bb00b679c32920217
I20260808 22:55:10.991668 6892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:11.003274 6892 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:11.003515 6892 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:11.007251 6892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: Generated new TSK 0
I20260808 22:55:11.007488 6892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:11.016750 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--builtin_ntp_servers=127.1.212.20:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:55:11.108484 6904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:11.108742 6904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:11.108769 6904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:11.110231 6904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:11.110291 6904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:11.110320 6904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:11.112082 6904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:11.112416 6904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:11.112824 6904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:11.118376 6909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:11.118665 6904 server_base.cc:1034] running on GCE node
W20260808 22:55:11.118503 6912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.118414 6910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:11.119025 6904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:11.119319 6904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:11.120517 6904 hybrid_clock.cc:648] HybridClock initialized: now 1786229711120500 us; error 39 us; skew 500 ppm
I20260808 22:55:11.122206 6904 webserver.cc:460] Webserver started at http://127.1.212.1:33029/ using document root <none> and password file <none>
I20260808 22:55:11.122499 6904 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:11.122582 6904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:11.122819 6904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:11.124058 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "861f96c2334c43859df4bd6069df1d6e"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "0bf87dd819d58229b04a2a1f8f562e6a"
server_key_iv: "41160e9910759b3b3fce2c739c5875de"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.124482 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "861f96c2334c43859df4bd6069df1d6e"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "0bf87dd819d58229b04a2a1f8f562e6a"
server_key_iv: "41160e9910759b3b3fce2c739c5875de"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.126111 6904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 22:55:11.127082 6918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.127286 6904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:11.127410 6904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "861f96c2334c43859df4bd6069df1d6e"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "0bf87dd819d58229b04a2a1f8f562e6a"
server_key_iv: "41160e9910759b3b3fce2c739c5875de"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.127501 6904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:11.131265 6904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:11.131585 6904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:11.131749 6904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:11.131994 6904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:11.132335 6904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:11.132391 6904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.132429 6904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:11.132476 6904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.139429 6904 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:37807
I20260808 22:55:11.139488 7039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:37807 every 8 connection(s)
I20260808 22:55:11.139895 6904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:11.143391 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.140855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:37063 (local address 127.1.212.62:40827)
0808 22:55:11.140922 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.140924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.141174 (+ 250us) server_negotiation.cc:407] Connection header received
0808 22:55:11.141757 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.141758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.141766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.141819 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.143107 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.143109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.143109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.143115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.143134 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.143139 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.143194 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.143293 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.143311 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:55:11.143657 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.140782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40827 (local address 127.1.212.1:37063)
0808 22:55:11.141046 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.141083 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.141624 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.141911 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.141915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.141924 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.142902 (+ 978us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.142902 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.142937 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.142973 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.143191 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.143198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.143202 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.143308 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.143384 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260808 22:55:11.144270 7040 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40827
I20260808 22:55:11.144408 7040 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:11.144586 7040 heartbeater.cc:511] Master 127.1.212.62:40827 requested a full tablet report, sending...
I20260808 22:55:11.144618 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 6904
I20260808 22:55:11.144701 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:11.144956 1872 external_mini_cluster.cc:1428] Setting key 21d257f233ffa8039a600035a57c0440
I20260808 22:55:11.145260 6834 ts_manager.cc:194] Registered new tserver with Master: 861f96c2334c43859df4bd6069df1d6e (127.1.212.1:37807)
I20260808 22:55:11.146261 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--builtin_ntp_servers=127.1.212.20:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:55:11.235025 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:11.235268 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:11.235294 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:11.236653 7043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:11.236716 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:11.236747 7043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:11.238396 7043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:11.238706 7043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:11.239061 7043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:11.244583 7050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.244609 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.244583 7051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:11.245000 7043 server_base.cc:1034] running on GCE node
I20260808 22:55:11.245239 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:11.245525 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:11.246685 7043 hybrid_clock.cc:648] HybridClock initialized: now 1786229711246674 us; error 34 us; skew 500 ppm
I20260808 22:55:11.248190 7043 webserver.cc:460] Webserver started at http://127.1.212.2:33959/ using document root <none> and password file <none>
I20260808 22:55:11.248481 7043 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:11.248538 7043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:11.248704 7043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:11.249917 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "6e387b2c1d2b41639acef2eb8912dc7f"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "7a19c08eac4ea012958cdcbaf60c95e8"
server_key_iv: "da4a4def347af6cab4a11e8b2e98cb99"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.250324 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "6e387b2c1d2b41639acef2eb8912dc7f"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "7a19c08eac4ea012958cdcbaf60c95e8"
server_key_iv: "da4a4def347af6cab4a11e8b2e98cb99"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.251781 7043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:11.252569 7061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.252821 7043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:11.252954 7043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "6e387b2c1d2b41639acef2eb8912dc7f"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "7a19c08eac4ea012958cdcbaf60c95e8"
server_key_iv: "da4a4def347af6cab4a11e8b2e98cb99"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.253039 7043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:11.256687 7043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:11.257040 7043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:11.257195 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:11.257426 7043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:11.257781 7043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:11.257815 7043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.257839 7043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:11.257885 7043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.264012 7043 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:41529
I20260808 22:55:11.264053 7188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:41529 every 8 connection(s)
I20260808 22:55:11.264429 7043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:11.267529 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.265239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:45007 (local address 127.1.212.62:40827)
0808 22:55:11.265300 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.265303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.265582 (+ 279us) server_negotiation.cc:407] Connection header received
0808 22:55:11.266076 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.266077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.266084 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.266139 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.267300 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.267302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.267302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.267306 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.267320 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.267325 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.267372 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.267449 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.267462 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:55:11.267717 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.265226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40827 (local address 127.1.212.2:45007)
0808 22:55:11.265444 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.265483 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.265981 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.266197 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.266200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.266209 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.267141 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.267142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.267175 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.267207 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.267377 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.267379 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.267381 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.267445 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.267498 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260808 22:55:11.268290 7189 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40827
I20260808 22:55:11.268414 7189 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:11.268582 7189 heartbeater.cc:511] Master 127.1.212.62:40827 requested a full tablet report, sending...
I20260808 22:55:11.269186 6833 ts_manager.cc:194] Registered new tserver with Master: 6e387b2c1d2b41639acef2eb8912dc7f (127.1.212.2:41529)
I20260808 22:55:11.272490 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7043
I20260808 22:55:11.272619 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:11.272854 1872 external_mini_cluster.cc:1428] Setting key 5033eaa486648a38bfa6f690dc26bfc2
I20260808 22:55:11.273941 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--builtin_ntp_servers=127.1.212.20:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:55:11.361824 7192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:11.362056 7192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:11.362082 7192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:11.363257 7192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:11.363299 7192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:11.363327 7192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:11.364686 7192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:11.364931 7192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:11.365204 7192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:11.370669 7202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.370751 7200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.370669 7198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:11.370893 7192 server_base.cc:1034] running on GCE node
I20260808 22:55:11.371098 7192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:11.371369 7192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:11.372615 7192 hybrid_clock.cc:648] HybridClock initialized: now 1786229711372587 us; error 32 us; skew 500 ppm
I20260808 22:55:11.374145 7192 webserver.cc:460] Webserver started at http://127.1.212.3:40779/ using document root <none> and password file <none>
I20260808 22:55:11.374425 7192 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:11.374504 7192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:11.374665 7192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:11.375833 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "38854f0f3b315a6629a56c663943ddd2"
server_key_iv: "28bac5c0cfc46d5a104e14a6140bc0a2"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.376242 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "38854f0f3b315a6629a56c663943ddd2"
server_key_iv: "28bac5c0cfc46d5a104e14a6140bc0a2"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.377846 7192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:11.378758 7209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.378975 7192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:11.379094 7192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "38854f0f3b315a6629a56c663943ddd2"
server_key_iv: "28bac5c0cfc46d5a104e14a6140bc0a2"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.379180 7192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:11.383275 7192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:11.383603 7192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:11.383767 7192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:11.384027 7192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:11.384424 7192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:11.384485 7192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.384529 7192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:11.384554 7192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.391804 7192 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:33859
I20260808 22:55:11.391846 7333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:33859 every 8 connection(s)
I20260808 22:55:11.392230 7192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:11.395766 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.393085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:55289 (local address 127.1.212.62:40827)
0808 22:55:11.393175 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.393177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.393418 (+ 241us) server_negotiation.cc:407] Connection header received
0808 22:55:11.393927 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.393928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.393934 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.393990 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.395250 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.395252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.395252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.395257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.395274 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.395278 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.395341 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.395617 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.395709 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:55:11.395780 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.393052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40827 (local address 127.1.212.3:55289)
0808 22:55:11.393287 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.393326 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.393833 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.394049 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.394052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.394061 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.395053 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.395053 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.395089 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.395125 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.395417 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.395426 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.395430 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.395521 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.395588 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260808 22:55:11.396428 7334 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40827
I20260808 22:55:11.396554 7334 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:11.396739 7334 heartbeater.cc:511] Master 127.1.212.62:40827 requested a full tablet report, sending...
I20260808 22:55:11.397217 6833 ts_manager.cc:194] Registered new tserver with Master: a85a04a8a7f54369a3dfdd9a55eab6e0 (127.1.212.3:33859)
I20260808 22:55:11.400341 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7192
I20260808 22:55:11.400467 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:11.400682 1872 external_mini_cluster.cc:1428] Setting key 12af6525111b704c038f464c1369f7f8
I20260808 22:55:11.401971 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:55:11.405189 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.403577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.1.212.62:40827)
0808 22:55:11.403646 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.403648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.403694 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:55:11.403798 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.403799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.403807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.403864 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.404849 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.404852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.404853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.404862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.404892 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.404897 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.404980 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.405110 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.405125 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:55:11.407354 6833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:11.409130 6833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:11.413934 7344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.411789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37807 (local address 127.0.0.1:35254)
0808 22:55:11.412030 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.412076 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.412141 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.412727 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.412735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.412743 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.413447 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.413448 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.413470 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.413493 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.413753 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.413761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.413764 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.413834 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.413851 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260808 22:55:11.413951 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.412046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.1.212.1:37807)
0808 22:55:11.412539 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.412545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.412558 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:11.412589 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.412592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.412604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.413066 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.413586 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.413595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.413596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.413600 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.413638 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.413668 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.413745 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.413852 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.413868 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":249,"threads_started":1}
I20260808 22:55:11.414220 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.412056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:33859 (local address 127.0.0.1:52494)
0808 22:55:11.412503 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.412517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.412577 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.413224 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.413226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.413229 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.413814 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.413815 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.413820 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.413829 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.414064 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.414065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.414065 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.414125 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.414141 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":95,"threads_started":1}
I20260808 22:55:11.414233 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.412290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.1.212.3:33859)
0808 22:55:11.412608 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.412613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.412624 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:11.412651 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.412653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.412663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.412737 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.413898 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.413908 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.413908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.413913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.413949 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.413980 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.414059 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.414134 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.414150 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260808 22:55:11.414510 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.411789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41529 (local address 127.0.0.1:54200)
0808 22:55:11.412316 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.412327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.412371 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.413207 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.413209 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.413213 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.413657 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.413658 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.413665 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.413679 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.414347 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.414348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.414348 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.414394 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.414411 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260808 22:55:11.414729 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.412046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.1.212.2:41529)
0808 22:55:11.412725 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.412730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.412741 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:11.412767 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.412769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.412804 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.412876 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.414184 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.414195 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.414196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.414200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.414237 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.414265 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.414320 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.414622 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.414642 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":246,"threads_started":1}
I20260808 22:55:11.415164 6970 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a (DEFAULT_TABLE table=test-table [id=064f6dfbf90244a2a9d03e4803317662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:11.415169 7260 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a (DEFAULT_TABLE table=test-table [id=064f6dfbf90244a2a9d03e4803317662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:11.415349 7115 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a (DEFAULT_TABLE table=test-table [id=064f6dfbf90244a2a9d03e4803317662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:11.415503 6970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0973cb9e584fd4a53d8c46259a0d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:11.415555 7260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0973cb9e584fd4a53d8c46259a0d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:11.415678 7115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0973cb9e584fd4a53d8c46259a0d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:11.420539 7350 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Bootstrap starting.
I20260808 22:55:11.420928 7349 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Bootstrap starting.
I20260808 22:55:11.421352 7351 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Bootstrap starting.
I20260808 22:55:11.421389 7350 tablet_bootstrap.cc:654] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:11.421746 7349 tablet_bootstrap.cc:654] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:11.421737 7350 log.cc:824] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:11.422128 7349 log.cc:824] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:11.422317 7351 tablet_bootstrap.cc:654] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:11.422619 7350 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: No bootstrap required, opened a new log
I20260808 22:55:11.422704 7350 ts_tablet_manager.cc:1397] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:55:11.422786 7351 log.cc:824] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:11.423085 7349 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: No bootstrap required, opened a new log
I20260808 22:55:11.423162 7349 ts_tablet_manager.cc:1397] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:55:11.425806 7350 raft_consensus.cc:348] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.426110 7350 raft_consensus.cc:374] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:11.426156 7350 raft_consensus.cc:729] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85a04a8a7f54369a3dfdd9a55eab6e0, State: Initialized, Role: FOLLOWER
I20260808 22:55:11.426347 7350 consensus_queue.cc:260] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.426698 7350 ts_tablet_manager.cc:1428] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 22:55:11.426683 7349 raft_consensus.cc:348] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.426882 7349 raft_consensus.cc:374] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:11.426990 7349 raft_consensus.cc:729] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e387b2c1d2b41639acef2eb8912dc7f, State: Initialized, Role: FOLLOWER
I20260808 22:55:11.427152 7349 consensus_queue.cc:260] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.427476 7349 ts_tablet_manager.cc:1428] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 22:55:11.427547 7189 heartbeater.cc:503] Master 127.1.212.62:40827 was elected leader, sending a full tablet report...
I20260808 22:55:11.428818 7351 tablet_bootstrap.cc:492] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: No bootstrap required, opened a new log
I20260808 22:55:11.428915 7351 ts_tablet_manager.cc:1397] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 22:55:11.430893 7334 heartbeater.cc:503] Master 127.1.212.62:40827 was elected leader, sending a full tablet report...
I20260808 22:55:11.431437 7351 raft_consensus.cc:348] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.431638 7351 raft_consensus.cc:374] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:11.431679 7351 raft_consensus.cc:729] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861f96c2334c43859df4bd6069df1d6e, State: Initialized, Role: FOLLOWER
I20260808 22:55:11.431806 7351 consensus_queue.cc:260] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.432168 7351 ts_tablet_manager.cc:1428] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:11.432292 7040 heartbeater.cc:503] Master 127.1.212.62:40827 was elected leader, sending a full tablet report...
W20260808 22:55:11.515086 7190 tablet.cc:2369] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:11.602983 7357 raft_consensus.cc:482] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:11.603183 7357 raft_consensus.cc:504] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.603644 7357 leader_election.cc:288] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 861f96c2334c43859df4bd6069df1d6e (127.1.212.1:37807), a85a04a8a7f54369a3dfdd9a55eab6e0 (127.1.212.3:33859)
I20260808 22:55:11.605192 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.603843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:33859 (local address 127.1.212.2:56799)
0808 22:55:11.604009 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.604021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.604066 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.604226 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.604227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.604230 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.604746 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.604747 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.604755 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.604768 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.605033 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.605034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.605035 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.605093 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.605106 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260808 22:55:11.605198 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.604016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:56799 (local address 127.1.212.3:33859)
0808 22:55:11.604079 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.604081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.604094 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:11.604129 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.604130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.604136 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.604202 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.604907 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.604909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.604909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.604934 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.604953 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.604958 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.605035 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.605090 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.605106 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:55:11.605311 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.603953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46965 (local address 127.1.212.1:37807)
0808 22:55:11.604011 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.604014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.604220 (+ 206us) server_negotiation.cc:407] Connection header received
0808 22:55:11.604251 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.604252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.604258 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.604314 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.605072 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.605074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.605074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.605079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.605100 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.605104 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.605162 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.605220 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.605234 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:55:11.605327 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.603846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37807 (local address 127.1.212.2:46965)
0808 22:55:11.604086 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.604096 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.604157 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.604343 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.604344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.604357 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:11.604959 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:11.604960 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:11.604965 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:11.604974 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:11.605169 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:11.605170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:11.605171 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:55:11.605221 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:55:11.605234 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":49,"thread_start_us":115,"threads_started":1}
I20260808 22:55:11.605576 7283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0973cb9e584fd4a53d8c46259a0d3a" candidate_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" is_pre_election: true
I20260808 22:55:11.605609 6993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0973cb9e584fd4a53d8c46259a0d3a" candidate_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "861f96c2334c43859df4bd6069df1d6e" is_pre_election: true
I20260808 22:55:11.605753 7283 raft_consensus.cc:2454] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e387b2c1d2b41639acef2eb8912dc7f in term 0.
I20260808 22:55:11.605753 6993 raft_consensus.cc:2454] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e387b2c1d2b41639acef2eb8912dc7f in term 0.
I20260808 22:55:11.606045 7065 leader_election.cc:302] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e387b2c1d2b41639acef2eb8912dc7f, a85a04a8a7f54369a3dfdd9a55eab6e0; no voters:
I20260808 22:55:11.606192 7357 raft_consensus.cc:2781] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:11.606262 7357 raft_consensus.cc:482] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:11.606300 7357 raft_consensus.cc:3037] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:11.618268 7357 raft_consensus.cc:504] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.618546 7357 leader_election.cc:288] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [CANDIDATE]: Term 1 election: Requested vote from peers 861f96c2334c43859df4bd6069df1d6e (127.1.212.1:37807), a85a04a8a7f54369a3dfdd9a55eab6e0 (127.1.212.3:33859)
I20260808 22:55:11.618870 7283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0973cb9e584fd4a53d8c46259a0d3a" candidate_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0"
I20260808 22:55:11.618883 6993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0973cb9e584fd4a53d8c46259a0d3a" candidate_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "861f96c2334c43859df4bd6069df1d6e"
I20260808 22:55:11.618992 7283 raft_consensus.cc:3037] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:11.619010 6993 raft_consensus.cc:3037] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:11.619894 7283 raft_consensus.cc:2454] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e387b2c1d2b41639acef2eb8912dc7f in term 1.
I20260808 22:55:11.620182 7065 leader_election.cc:302] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e387b2c1d2b41639acef2eb8912dc7f, a85a04a8a7f54369a3dfdd9a55eab6e0; no voters:
I20260808 22:55:11.620285 6993 raft_consensus.cc:2454] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e387b2c1d2b41639acef2eb8912dc7f in term 1.
I20260808 22:55:11.620326 7357 raft_consensus.cc:2781] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:11.620507 7357 raft_consensus.cc:686] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 1 LEADER]: Becoming Leader. State: Replica: 6e387b2c1d2b41639acef2eb8912dc7f, State: Running, Role: LEADER
I20260808 22:55:11.620642 7357 consensus_queue.cc:237] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } }
I20260808 22:55:11.621644 6833 catalog_manager.cc:5495] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e387b2c1d2b41639acef2eb8912dc7f (127.1.212.2). New cstate: current_term: 1 leader_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e387b2c1d2b41639acef2eb8912dc7f" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260808 22:55:11.640858 7041 tablet.cc:2369] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:55:11.642798 7336 tablet.cc:2369] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:11.652241 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.651093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.1.212.2:41529)
0808 22:55:11.651165 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.651167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.651191 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:55:11.651244 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.651245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.651262 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.651323 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.651934 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.651936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.651937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.651943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.651965 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.651970 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.652040 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.652136 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.652152 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:11.654506 7283 raft_consensus.cc:1261] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 1 FOLLOWER]: Refusing update from remote peer 6e387b2c1d2b41639acef2eb8912dc7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:11.654788 6993 raft_consensus.cc:1261] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 1 FOLLOWER]: Refusing update from remote peer 6e387b2c1d2b41639acef2eb8912dc7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:11.655007 7361 consensus_queue.cc:1035] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85a04a8a7f54369a3dfdd9a55eab6e0" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:11.655247 7357 consensus_queue.cc:1035] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "861f96c2334c43859df4bd6069df1d6e" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:11.659036 7365 mvcc.cc:205] Tried to move back new op lower bound from 7316396898933477376 to 7316396898798854144. Current Snapshot: MvccSnapshot[applied={T|T < 7316396898933477376}]
I20260808 22:55:11.659922 7366 mvcc.cc:205] Tried to move back new op lower bound from 7316396898933477376 to 7316396898798854144. Current Snapshot: MvccSnapshot[applied={T|T < 7316396898933477376 or (T in {7316396898933477376})}]
I20260808 22:55:11.660758 6834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:11.660910 6834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:11.661823 6834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abb743ce7dec4588b10cd84c9a171be4: Sending DeleteTablet for 3 replicas of tablet 2d0973cb9e584fd4a53d8c46259a0d3a
I20260808 22:55:11.662256 7260 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:11 UTC) from {username='slave'} at 127.0.0.1:52494
I20260808 22:55:11.662320 7115 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:11 UTC) from {username='slave'} at 127.0.0.1:54200
I20260808 22:55:11.662328 6970 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0973cb9e584fd4a53d8c46259a0d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:11 UTC) from {username='slave'} at 127.0.0.1:35254
I20260808 22:55:11.662568 7374 tablet_replica.cc:330] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: stopping tablet replica
I20260808 22:55:11.662616 7376 tablet_replica.cc:330] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e: stopping tablet replica
I20260808 22:55:11.662738 7374 raft_consensus.cc:2229] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:11.662838 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6904
I20260808 22:55:11.662873 7374 raft_consensus.cc:2258] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:11.662964 7376 raft_consensus.cc:2229] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:11.663059 7375 tablet_replica.cc:330] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: stopping tablet replica
I20260808 22:55:11.663182 7375 raft_consensus.cc:2229] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:11.663201 7376 raft_consensus.cc:2258] T 2d0973cb9e584fd4a53d8c46259a0d3a P 861f96c2334c43859df4bd6069df1d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:11.663304 7375 raft_consensus.cc:2258] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:11.663491 7374 ts_tablet_manager.cc:1905] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:11.663870 7375 ts_tablet_manager.cc:1905] T 2d0973cb9e584fd4a53d8c46259a0d3a P 6e387b2c1d2b41639acef2eb8912dc7f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:55:11.671711 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:37807 (error 108)
I20260808 22:55:11.672729 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.672516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37807 (local address 127.0.0.1:35268)
0808 22:55:11.672620 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.672679 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 22:55:11.672981 6818 catalog_manager.cc:4584] TS 861f96c2334c43859df4bd6069df1d6e (127.1.212.1:37807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d0973cb9e584fd4a53d8c46259a0d3a: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37807: connect: Connection refused (error 111)
I20260808 22:55:11.673197 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7043
I20260808 22:55:11.722203 7374 ts_tablet_manager.cc:1918] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:11.722354 7374 log.cc:1196] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/2d0973cb9e584fd4a53d8c46259a0d3a
I20260808 22:55:11.722654 7374 ts_tablet_manager.cc:1939] T 2d0973cb9e584fd4a53d8c46259a0d3a P a85a04a8a7f54369a3dfdd9a55eab6e0: Deleting consensus metadata
I20260808 22:55:11.725791 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.725589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37807 (local address 127.0.0.1:35272)
0808 22:55:11.725681 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.725733 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 22:55:11.726402 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.726239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37807 (local address 127.0.0.1:35274)
0808 22:55:11.726302 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.726363 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:55:11.726738 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.726644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41529 (local address 127.0.0.1:54220)
0808 22:55:11.726681 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.726705 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.2:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 22:55:11.726914 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7192
I20260808 22:55:11.743598 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 6800
2026-08-08T22:55:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (916 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 22:55:11.767904 1872 mini_oidc.cc:152] Starting JWKS server
I20260808 22:55:11.769338 1872 webserver.cc:460] Webserver started at https://127.0.0.1:42787/ using document root <none> and password file <none>
I20260808 22:55:11.769433 1872 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:55:11.769575 1872 webserver.cc:460] Webserver started at http://127.0.0.1:35567/ using document root <none> and password file <none>
2026-08-08T22:55:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:11Z Disabled control of system clock
I20260808 22:55:11.780275 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36035
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:36035
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42787/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/testchainca.cert with env {}
W20260808 22:55:11.872308 7383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:11.872574 7383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:11.872601 7383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:11.873888 7383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:11.873940 7383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:11.873953 7383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:11.873967 7383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:11.873980 7383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:55:11.873993 7383 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:55:11.874006 7383 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42787/jwks.json
I20260808 22:55:11.875986 7383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:36035
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42787/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:11.876319 7383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:11.876543 7383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:11.881857 7391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.881857 7389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:11.881878 7388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:11.882282 7383 server_base.cc:1034] running on GCE node
I20260808 22:55:11.882558 7383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:11.883160 7383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:11.884311 7383 hybrid_clock.cc:648] HybridClock initialized: now 1786229711884295 us; error 44 us; skew 500 ppm
I20260808 22:55:11.885799 7383 webserver.cc:460] Webserver started at http://127.1.212.62:43083/ using document root <none> and password file <none>
I20260808 22:55:11.886080 7383 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:11.886164 7383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:11.886368 7383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:11.887547 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "e357caa7be284fdab007f2e05238fdd9"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "4d303de232fe23bf504e61fbfd272841"
server_key_iv: "9b279dabd080d0bc8731f7572ea63f8a"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.887933 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "e357caa7be284fdab007f2e05238fdd9"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "4d303de232fe23bf504e61fbfd272841"
server_key_iv: "9b279dabd080d0bc8731f7572ea63f8a"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.889477 7383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 22:55:11.890450 7399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:11.890630 7383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:11.890744 7383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "e357caa7be284fdab007f2e05238fdd9"
format_stamp: "Formatted at 2026-08-08 22:55:11 on dist-test-slave-1x4g"
server_key: "4d303de232fe23bf504e61fbfd272841"
server_key_iv: "9b279dabd080d0bc8731f7572ea63f8a"
server_key_version: "encryptionkey@0"
I20260808 22:55:11.891073 7383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:11.898872 7383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:11.899531 7383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:11.899709 7383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:11.904258 7383 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:36035
I20260808 22:55:11.904296 7459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:36035 every 8 connection(s)
I20260808 22:55:11.904726 7383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:11.905884 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:11.906843 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7383
I20260808 22:55:11.906942 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:11.907142 1872 external_mini_cluster.cc:1428] Setting key 671a17c818d409957a644bd1d70d026b
I20260808 22:55:11.909580 7460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: Bootstrap starting.
I20260808 22:55:11.910392 7460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:11.910732 7460 log.cc:824] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:11.911609 7460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: No bootstrap required, opened a new log
I20260808 22:55:11.913702 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.907727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.1.212.62:36035)
0808 22:55:11.907955 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.907965 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.908000 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:55:11.908583 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.908594 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.908688 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.908870 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:11.909120 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.910481 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.911367 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.911995 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.912131 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:11.912887 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:11.912892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:11.912892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:11.913116 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:11.913161 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:11.913177 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:11.913251 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:11.913320 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:55:11.913417 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260808 22:55:11.914149 7460 raft_consensus.cc:348] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } }
I20260808 22:55:11.914340 7460 raft_consensus.cc:374] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:11.914389 7460 raft_consensus.cc:729] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e357caa7be284fdab007f2e05238fdd9, State: Initialized, Role: FOLLOWER
I20260808 22:55:11.914525 7460 consensus_queue.cc:260] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } }
I20260808 22:55:11.914618 7460 raft_consensus.cc:388] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:11.914669 7460 raft_consensus.cc:482] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:11.914718 7460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:11.914913 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:55:11.927932 7460 raft_consensus.cc:504] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } }
I20260808 22:55:11.928185 7460 leader_election.cc:302] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e357caa7be284fdab007f2e05238fdd9; no voters:
I20260808 22:55:11.928467 7460 leader_election.cc:288] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:11.928678 7468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:11.928833 7468 raft_consensus.cc:686] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [term 1 LEADER]: Becoming Leader. State: Replica: e357caa7be284fdab007f2e05238fdd9, State: Running, Role: LEADER
I20260808 22:55:11.928838 7460 sys_catalog.cc:564] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:11.928977 7468 consensus_queue.cc:237] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } }
I20260808 22:55:11.929364 7471 sys_catalog.cc:455] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e357caa7be284fdab007f2e05238fdd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } } }
I20260808 22:55:11.929430 7468 sys_catalog.cc:455] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e357caa7be284fdab007f2e05238fdd9. Latest consensus state: current_term: 1 leader_uuid: "e357caa7be284fdab007f2e05238fdd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e357caa7be284fdab007f2e05238fdd9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36035 } } }
I20260808 22:55:11.929586 7468 sys_catalog.cc:458] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:11.929802 7471 sys_catalog.cc:458] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9 [sys.catalog]: This master's current role is: LEADER
W20260808 22:55:11.930994 7486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:11.931607 7486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:11.931711 7477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:11.931907 7477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:11.933553 7477 catalog_manager.cc:1347] Generated new cluster ID: e28cfcb9a5fe40e2b47c448f5b1ca785
I20260808 22:55:11.933614 7477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:11.971189 7477 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:11.971978 7477 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:11.975845 7477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e357caa7be284fdab007f2e05238fdd9: Generated new TSK 0
I20260808 22:55:11.976137 7477 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:55:12.005441 7495 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 22:55:12.005630 7495 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:55:12.005861 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:11.988735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.1.212.62:36035)
0808 22:55:11.988817 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:11.988820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:11.989001 (+ 181us) server_negotiation.cc:407] Connection header received
0808 22:55:11.989121 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:11.989122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:11.989133 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:11.989209 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:55:11.989458 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.990018 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.991268 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.991503 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.991567 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:11.991852 (+ 285us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:55:12.005556 (+ 13704us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:55:12.005780 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 22:55:12.005966 7465 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 22:55:12.006628 7494 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:11.988659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:36035 (local address 127.0.0.1:35238)
0808 22:55:11.988912 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:11.988926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:11.989013 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:11.989252 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:11.989253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:11.989255 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:55:11.989350 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:11.989352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.990119 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.990120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:11.991144 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:11.991149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:11.991598 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:11.991600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:11.991676 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:11.991710 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:55:12.005868 (+ 14158us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:55:12.006130 (+ 262us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.212.62:36035: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":148,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.212.62:36035: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 22:55:12.007164 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7383
2026-08-08T22:55:12Z chronyd exiting
I20260808 22:55:12.046592 1872 test_util.cc:175] -----------------------------------------------
I20260808 22:55:12.046684 1872 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229688278322-1872-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (280 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 22:55:12.047576 1872 mini_oidc.cc:152] Starting JWKS server
I20260808 22:55:12.048307 1872 webserver.cc:460] Webserver started at https://127.0.0.1:46125/ using document root <none> and password file <none>
I20260808 22:55:12.048382 1872 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:55:12.048548 1872 webserver.cc:460] Webserver started at http://127.0.0.1:42779/ using document root <none> and password file <none>
2026-08-08T22:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:12Z Disabled control of system clock
I20260808 22:55:12.059016 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:42151
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:41103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:42151
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46125/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/testchainca.cert with env {}
W20260808 22:55:12.149328 7501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:12.149597 7501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:12.149641 7501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:12.151022 7501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:12.151335 7501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:12.151427 7501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:12.151460 7501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:12.151490 7501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:55:12.151522 7501 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:55:12.151549 7501 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46125/jwks.json
I20260808 22:55:12.153493 7501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:42151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:42151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46125/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:12.153817 7501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:12.154085 7501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:12.159806 7507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.159828 7510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.159830 7506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:12.160053 7501 server_base.cc:1034] running on GCE node
I20260808 22:55:12.160396 7501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:12.160823 7501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:12.162010 7501 hybrid_clock.cc:648] HybridClock initialized: now 1786229712162005 us; error 52 us; skew 500 ppm
I20260808 22:55:12.163532 7501 webserver.cc:460] Webserver started at http://127.1.212.62:34797/ using document root <none> and password file <none>
I20260808 22:55:12.163836 7501 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:12.163913 7501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:12.164088 7501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:12.165346 7501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "36d5968b58ef4608b4a932c208990945"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "8ac7a93b65163141ec66504bd609b000"
server_key_iv: "8223eac3f4e31d24ac21ee096f3eee18"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.165729 7501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "36d5968b58ef4608b4a932c208990945"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "8ac7a93b65163141ec66504bd609b000"
server_key_iv: "8223eac3f4e31d24ac21ee096f3eee18"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.167238 7501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:12.168076 7518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:12.168289 7501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:12.168413 7501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "36d5968b58ef4608b4a932c208990945"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "8ac7a93b65163141ec66504bd609b000"
server_key_iv: "8223eac3f4e31d24ac21ee096f3eee18"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.168498 7501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:12.190397 7501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:12.190789 7501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:12.190963 7501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:12.195345 7501 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:42151
I20260808 22:55:12.195405 7577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:42151 every 8 connection(s)
I20260808 22:55:12.195829 7501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:12.197136 7578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:12.199987 7578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945: Bootstrap starting.
I20260808 22:55:12.200634 7578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:12.200982 7578 log.cc:824] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:12.201821 7578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945: No bootstrap required, opened a new log
I20260808 22:55:12.203557 7578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } }
I20260808 22:55:12.203742 7578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:12.203766 7578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d5968b58ef4608b4a932c208990945, State: Initialized, Role: FOLLOWER
I20260808 22:55:12.203884 7578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } }
I20260808 22:55:12.203939 7578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:12.203962 7578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:12.203990 7578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:12.204972 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7501
I20260808 22:55:12.205094 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:12.205287 1872 external_mini_cluster.cc:1428] Setting key a0ed83114f3c1b6bc64c7a61fc239a2a
I20260808 22:55:12.211249 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.205903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.1.212.62:42151)
0808 22:55:12.206218 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.206227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.206256 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:55:12.206380 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.206386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.206690 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.206850 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:12.207132 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.208079 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.208975 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.209454 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.209615 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.210408 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:12.210419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:12.210419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:12.210612 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:12.210651 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:12.210665 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:12.210740 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:12.210817 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:55:12.210911 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260808 22:55:12.212352 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:55:12.234972 7578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } }
I20260808 22:55:12.235248 7578 leader_election.cc:302] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36d5968b58ef4608b4a932c208990945; no voters:
I20260808 22:55:12.235586 7578 leader_election.cc:288] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:12.235635 7584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:12.235769 7584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [term 1 LEADER]: Becoming Leader. State: Replica: 36d5968b58ef4608b4a932c208990945, State: Running, Role: LEADER
I20260808 22:55:12.235908 7578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:12.235859 7584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } }
I20260808 22:55:12.236258 7584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d5968b58ef4608b4a932c208990945. Latest consensus state: current_term: 1 leader_uuid: "36d5968b58ef4608b4a932c208990945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } } }
I20260808 22:55:12.236370 7584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.236274 7585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36d5968b58ef4608b4a932c208990945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d5968b58ef4608b4a932c208990945" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42151 } } }
I20260808 22:55:12.236418 7585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.237361 7594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:12.238001 7594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:12.239981 7594 catalog_manager.cc:1347] Generated new cluster ID: 224ca4647f1c4fea8ac0e5190d0705e9
I20260808 22:55:12.240048 7594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:12.251683 7594 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:12.252285 7594 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:12.266222 7594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36d5968b58ef4608b4a932c208990945: Generated new TSK 0
I20260808 22:55:12.266455 7594 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:55:12.303869 7615 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 22:55:12.303975 7615 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:55:12.304299 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.282103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.1.212.62:42151)
0808 22:55:12.282485 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.282487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.282494 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:55:12.282533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.282534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.282557 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.282651 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:55:12.282821 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.283333 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.284227 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.284444 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.284526 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.284736 (+ 210us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:55:12.304019 (+ 19283us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46125/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:55:12.304218 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39696: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46125/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":358}
W20260808 22:55:12.304389 7583 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39696: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46125/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 22:55:12.304462 7614 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:12.282019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:42151 (local address 127.0.0.1:39696)
0808 22:55:12.282229 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:12.282240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:12.282311 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:12.282657 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:12.282658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:12.282660 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:55:12.282747 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.282749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.283460 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.283461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.284126 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.284130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.284531 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.284533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.284599 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.284608 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:55:12.304222 (+ 19614us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46125/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:55:12.304348 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.212.62:42151: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46125/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":121,"threads_started":1}
I20260808 22:55:12.304741 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7501
2026-08-08T22:55:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (299 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 22:55:12.346884 1872 mini_oidc.cc:152] Starting JWKS server
I20260808 22:55:12.347676 1872 webserver.cc:460] Webserver started at https://127.0.0.1:42317/ using document root <none> and password file <none>
I20260808 22:55:12.347754 1872 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:55:12.347937 1872 webserver.cc:460] Webserver started at http://127.0.0.1:37549/ using document root <none> and password file <none>
2026-08-08T22:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:12Z Disabled control of system clock
I20260808 22:55:12.358240 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:38545
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:38545
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42317/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:55:12.449707 7621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:12.449970 7621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:12.450011 7621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:12.451333 7621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:12.451401 7621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:12.451426 7621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:12.451462 7621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:12.451484 7621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:55:12.451516 7621 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:55:12.451542 7621 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42317/jwks.json
I20260808 22:55:12.453685 7621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:38545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42317/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:12.454018 7621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:12.454392 7621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:12.459674 7626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.459669 7631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.459697 7627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:12.459971 7621 server_base.cc:1034] running on GCE node
I20260808 22:55:12.460242 7621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:12.460582 7621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:12.461735 7621 hybrid_clock.cc:648] HybridClock initialized: now 1786229712461711 us; error 43 us; skew 500 ppm
I20260808 22:55:12.463162 7621 webserver.cc:460] Webserver started at http://127.1.212.62:32919/ using document root <none> and password file <none>
I20260808 22:55:12.463465 7621 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:12.463543 7621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:12.463696 7621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:12.464988 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "72ce5b4a57be4fd6aaae3587106d27e9"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "2e2091aef95f7eda83ea9b6220a1016f"
server_key_iv: "75b101cf8c5474e277818ff2a2c9620a"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.465376 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "72ce5b4a57be4fd6aaae3587106d27e9"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "2e2091aef95f7eda83ea9b6220a1016f"
server_key_iv: "75b101cf8c5474e277818ff2a2c9620a"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.466822 7621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:12.467672 7637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:12.467883 7621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:12.468086 7621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "72ce5b4a57be4fd6aaae3587106d27e9"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "2e2091aef95f7eda83ea9b6220a1016f"
server_key_iv: "75b101cf8c5474e277818ff2a2c9620a"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.468178 7621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:12.485299 7621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:12.485630 7621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:12.485746 7621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:12.490113 7621 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:38545
I20260808 22:55:12.490209 7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:38545 every 8 connection(s)
I20260808 22:55:12.490563 7621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:12.491438 7696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:12.494347 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7621
I20260808 22:55:12.494465 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:12.494647 1872 external_mini_cluster.cc:1428] Setting key 040abb84d37554f0a9c0b1480a8b2b45
I20260808 22:55:12.495242 7696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9: Bootstrap starting.
I20260808 22:55:12.495904 7696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:12.496227 7696 log.cc:824] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:12.497175 7696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9: No bootstrap required, opened a new log
I20260808 22:55:12.499347 7696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } }
I20260808 22:55:12.499601 7696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:12.499678 7696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ce5b4a57be4fd6aaae3587106d27e9, State: Initialized, Role: FOLLOWER
I20260808 22:55:12.499851 7696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } }
I20260808 22:55:12.499953 7696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:12.499980 7696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:12.500068 7696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:12.500851 7699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.495251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.1.212.62:38545)
0808 22:55:12.495490 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.495498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.495524 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:55:12.495965 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.495971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.496077 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.496236 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:12.496459 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.497716 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.498603 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.499063 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.499228 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.500087 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:12.500091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:12.500091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:12.500311 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:12.500348 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:12.500360 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:12.500445 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:12.500517 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:55:12.500608 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260808 22:55:12.502177 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:55:12.537946 7696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } }
I20260808 22:55:12.538188 7696 leader_election.cc:302] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72ce5b4a57be4fd6aaae3587106d27e9; no voters:
I20260808 22:55:12.538441 7696 leader_election.cc:288] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:12.538563 7701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:12.538753 7701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [term 1 LEADER]: Becoming Leader. State: Replica: 72ce5b4a57be4fd6aaae3587106d27e9, State: Running, Role: LEADER
I20260808 22:55:12.538841 7696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:12.538865 7701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } }
I20260808 22:55:12.539318 7702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } } }
I20260808 22:55:12.539407 7702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.539649 7703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72ce5b4a57be4fd6aaae3587106d27e9. Latest consensus state: current_term: 1 leader_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72ce5b4a57be4fd6aaae3587106d27e9" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38545 } } }
I20260808 22:55:12.539994 7703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.540282 7709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:12.540879 7709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:12.542742 7709 catalog_manager.cc:1347] Generated new cluster ID: bd84c51d7ab9407a9cd9433522eeb4c1
I20260808 22:55:12.542805 7709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:12.552717 7709 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:12.553514 7709 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:12.556607 7709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72ce5b4a57be4fd6aaae3587106d27e9: Generated new TSK 0
I20260808 22:55:12.556843 7709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:12.570232 7699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.566459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.1.212.62:38545)
0808 22:55:12.566505 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.566508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.566638 (+ 130us) server_negotiation.cc:407] Connection header received
0808 22:55:12.566736 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.566737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.566749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.566838 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:55:12.567018 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.567539 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.568396 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.568619 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.568707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.568938 (+ 231us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:55:12.569990 (+ 1052us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42317/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:55:12.570133 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52668: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42317/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 22:55:12.570300 7699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52668: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42317/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 22:55:12.572548 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:12.566390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:38545 (local address 127.0.0.1:52668)
0808 22:55:12.566556 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:12.566568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:12.566660 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:12.566852 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:12.566853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:12.566855 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:55:12.566919 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.566921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.567653 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.567654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.568295 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.568297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.568720 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.568722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.568810 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.568817 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:55:12.570394 (+ 1577us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42317/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:55:12.572429 (+ 2035us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.212.62:38545: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42317/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1788,"thread_start_us":110,"threads_started":1}
I20260808 22:55:12.572918 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7621
E20260808 22:55:12.582171 7729 webserver.cc:533] Webserver: sslize failed:
E20260808 22:55:12.582284 7729 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T22:55:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (266 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T22:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:12Z Disabled control of system clock
I20260808 22:55:12.623113 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34027
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34027
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/testchainca.cert with env {}
W20260808 22:55:12.710213 7734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:12.710496 7734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:12.710526 7734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:12.712073 7734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:12.712146 7734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:12.712162 7734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:12.712174 7734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:12.712188 7734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:55:12.712249 7734 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:55:12.712276 7734 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 22:55:12.714268 7734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:12.714632 7734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:12.714911 7734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:12.721213 7739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.721231 7745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:12.721225 7742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:12.721535 7734 server_base.cc:1034] running on GCE node
I20260808 22:55:12.721832 7734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:12.722241 7734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:12.723445 7734 hybrid_clock.cc:648] HybridClock initialized: now 1786229712723419 us; error 45 us; skew 500 ppm
I20260808 22:55:12.725201 7734 webserver.cc:460] Webserver started at http://127.1.212.62:43949/ using document root <none> and password file <none>
I20260808 22:55:12.725509 7734 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:12.725564 7734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:12.725721 7734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:12.726974 7734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f4e271b3ba5f4459a3997f0aee842283"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "b18d479f61fbe39904dad7cd0ca046f8"
server_key_iv: "0bf42f66df26175a5c197ee82b1dbcd8"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.727384 7734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f4e271b3ba5f4459a3997f0aee842283"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "b18d479f61fbe39904dad7cd0ca046f8"
server_key_iv: "0bf42f66df26175a5c197ee82b1dbcd8"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.728942 7734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:12.729928 7751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:12.730140 7734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:12.730258 7734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f4e271b3ba5f4459a3997f0aee842283"
format_stamp: "Formatted at 2026-08-08 22:55:12 on dist-test-slave-1x4g"
server_key: "b18d479f61fbe39904dad7cd0ca046f8"
server_key_iv: "0bf42f66df26175a5c197ee82b1dbcd8"
server_key_version: "encryptionkey@0"
I20260808 22:55:12.730343 7734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:12.747103 7734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:12.747561 7734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:12.747742 7734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:12.752934 7734 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:34027
I20260808 22:55:12.752987 7812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:34027 every 8 connection(s)
I20260808 22:55:12.753366 7734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:12.754459 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:12.757136 7813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283: Bootstrap starting.
I20260808 22:55:12.757803 7813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:12.758143 7813 log.cc:824] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:12.758922 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7734
I20260808 22:55:12.758997 7813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283: No bootstrap required, opened a new log
I20260808 22:55:12.759073 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:12.759526 1872 external_mini_cluster.cc:1428] Setting key 9ba76db54bd1c9b32ef0fde7268a6cd2
I20260808 22:55:12.761310 7813 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } }
I20260808 22:55:12.761504 7813 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:12.761546 7813 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e271b3ba5f4459a3997f0aee842283, State: Initialized, Role: FOLLOWER
I20260808 22:55:12.761696 7813 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } }
I20260808 22:55:12.761806 7813 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:12.761853 7813 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:12.761909 7813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:12.765976 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.760207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.1.212.62:34027)
0808 22:55:12.760422 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.760430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.760457 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:55:12.761274 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.761283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.761360 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.761514 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:12.761767 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.763098 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.763912 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.764422 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.764524 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.765243 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:12.765247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:12.765248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:12.765437 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:12.765508 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:12.765527 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:12.765583 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:12.765669 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 22:55:12.765765 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260808 22:55:12.767136 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:55:12.848541 7813 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } }
I20260808 22:55:12.848829 7813 leader_election.cc:302] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4e271b3ba5f4459a3997f0aee842283; no voters:
I20260808 22:55:12.849138 7813 leader_election.cc:288] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:12.849223 7819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:12.849432 7819 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [term 1 LEADER]: Becoming Leader. State: Replica: f4e271b3ba5f4459a3997f0aee842283, State: Running, Role: LEADER
I20260808 22:55:12.849483 7813 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:12.849555 7819 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } }
I20260808 22:55:12.850033 7820 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4e271b3ba5f4459a3997f0aee842283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } } }
I20260808 22:55:12.850133 7820 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.850399 7821 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4e271b3ba5f4459a3997f0aee842283. Latest consensus state: current_term: 1 leader_uuid: "f4e271b3ba5f4459a3997f0aee842283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e271b3ba5f4459a3997f0aee842283" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34027 } } }
I20260808 22:55:12.850808 7821 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:12.851022 7826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:12.851706 7826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:12.853698 7826 catalog_manager.cc:1347] Generated new cluster ID: e0bdc5f3a74f437da06c81f706455207
I20260808 22:55:12.853766 7826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:12.867550 7826 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:12.868157 7826 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:12.880828 7826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4e271b3ba5f4459a3997f0aee842283: Generated new TSK 0
I20260808 22:55:12.881102 7826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:13.718559 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:12.889975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.1.212.62:34027)
0808 22:55:12.890044 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:12.890046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:12.890076 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:55:12.890173 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:12.890175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:12.890187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:12.890257 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:55:12.890437 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.890956 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.892149 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.892332 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.892384 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.892604 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:55:13.718306 (+825702us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:55:13.718493 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58670: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 22:55:13.718626 7818 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58670: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 22:55:13.718786 7845 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:12.889837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34027 (local address 127.0.0.1:58670)
0808 22:55:12.890007 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:12.890021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:12.890080 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:12.890285 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:12.890286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:12.890289 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:55:12.890361 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.890364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.891057 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.891058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.892049 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:12.892052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:12.892418 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:12.892419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:12.892493 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:12.892500 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:55:13.718484 (+825984us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:55:13.718611 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.212.62:34027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260808 22:55:13.719177 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7734
2026-08-08T22:55:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1126 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (100 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:13 dist-test-slave-1x4g krb5kdc[7857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:13 dist-test-slave-1x4g krb5kdc[7857](info): set up 2 sockets
Aug 08 22:55:13 dist-test-slave-1x4g krb5kdc[7857](info): commencing operation
krb5kdc: starting...
W20260808 22:55:14.994345 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:15.137315 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40753
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:15.225138 7871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:15.225373 7871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:15.225399 7871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:15.226594 7871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:15.226634 7871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:15.226648 7871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:15.226660 7871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:15.226672 7871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:15.228204 7871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:15.228442 7871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:15.228637 7871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:15.233937 7878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.233949 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:15.234122 7871 server_base.cc:1034] running on GCE node
W20260808 22:55:15.234184 7877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:15.234522 7871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:15.234871 7871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:15.236065 7871 hybrid_clock.cc:648] HybridClock initialized: now 1786229715236059 us; error 41 us; skew 500 ppm
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.242449 7871 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:15.242997 7871 webserver.cc:460] Webserver started at http://127.1.212.62:39225/ using document root <none> and password file <none>
I20260808 22:55:15.243294 7871 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:15.243381 7871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:15.243557 7871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:15.244848 7871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "65787793213946aeadde7c730bc6d54b"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "e740d4c982a1ab6a8cdf2b2cac226ef1"
server_key_iv: "8f83ce9c4f078e6acd9f5ad42e71bd9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.245266 7871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "65787793213946aeadde7c730bc6d54b"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "e740d4c982a1ab6a8cdf2b2cac226ef1"
server_key_iv: "8f83ce9c4f078e6acd9f5ad42e71bd9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.246809 7871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:15.247746 7888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.247997 7871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:15.248142 7871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "65787793213946aeadde7c730bc6d54b"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "e740d4c982a1ab6a8cdf2b2cac226ef1"
server_key_iv: "8f83ce9c4f078e6acd9f5ad42e71bd9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.248235 7871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:15.262050 7871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:15.264478 7871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:15.264674 7871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:15.269085 7871 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:40753
I20260808 22:55:15.269160 7946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:40753 every 8 connection(s)
I20260808 22:55:15.269603 7871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:15.270659 7947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:15.273341 7947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b: Bootstrap starting.
I20260808 22:55:15.273959 7947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:15.274273 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7871
I20260808 22:55:15.274394 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:15.274401 7947 log.cc:824] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:15.274608 1872 external_mini_cluster.cc:1428] Setting key cd6afee3a88b8140a6f50106860844db
I20260808 22:55:15.275516 7947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b: No bootstrap required, opened a new log
I20260808 22:55:15.278590 7947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } }
I20260808 22:55:15.278831 7947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:15.278873 7947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65787793213946aeadde7c730bc6d54b, State: Initialized, Role: FOLLOWER
I20260808 22:55:15.279021 7947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } }
I20260808 22:55:15.279106 7947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:15.279145 7947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:15.279202 7947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:15.280018 7947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } }
I20260808 22:55:15.280171 7947 leader_election.cc:302] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65787793213946aeadde7c730bc6d54b; no voters:
I20260808 22:55:15.280413 7947 leader_election.cc:288] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:15.280751 7952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:15.280828 7947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:15.281060 7952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [term 1 LEADER]: Becoming Leader. State: Replica: 65787793213946aeadde7c730bc6d54b, State: Running, Role: LEADER
I20260808 22:55:15.281234 7952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } }
I20260808 22:55:15.281850 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65787793213946aeadde7c730bc6d54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } } }
I20260808 22:55:15.281966 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:15.282197 7954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65787793213946aeadde7c730bc6d54b. Latest consensus state: current_term: 1 leader_uuid: "65787793213946aeadde7c730bc6d54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65787793213946aeadde7c730bc6d54b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40753 } } }
I20260808 22:55:15.282270 7954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:15.282352 7961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:15.283098 7961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:15.285072 7961 catalog_manager.cc:1347] Generated new cluster ID: 14c5930051fb425a8c3094b8d0323961
I20260808 22:55:15.285143 7961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:15.288700 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.275284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.1.212.62:40753)
0808 22:55:15.275454 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.275464 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.275502 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:55:15.276061 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.276070 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.276126 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.276247 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.277250 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.278177 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.279278 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.279753 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.279856 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.283767 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.283772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.283772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.283787 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.286223 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.286737 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.286741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.286741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.286870 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.287113 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.287114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.287114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.287486 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.287660 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.288264 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.288445 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260808 22:55:15.293327 7961 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:15.294159 7961 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:15.299434 7961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65787793213946aeadde7c730bc6d54b: Generated new TSK 0
I20260808 22:55:15.299738 7961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:15.337600 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:15.435874 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:15.436113 7976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:15.436141 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:15.437493 7976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:15.437548 7976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:15.437580 7976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:15.439363 7976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:15.439666 7976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:15.439934 7976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:15.445242 7982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.445266 7984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.445251 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:15.445729 7976 server_base.cc:1034] running on GCE node
I20260808 22:55:15.445967 7976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:15.446228 7976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:15.447436 7976 hybrid_clock.cc:648] HybridClock initialized: now 1786229715447426 us; error 39 us; skew 500 ppm
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.450373 7976 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:15.450927 7976 webserver.cc:460] Webserver started at http://127.1.212.1:39733/ using document root <none> and password file <none>
I20260808 22:55:15.451172 7976 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:15.451221 7976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:15.451319 7976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:15.452613 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "e3109952d147441fa6f70bd8b431d958"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6fda72fca7e4c6d20cbc58e357d1286b"
server_key_iv: "ecb37e6951461ed0d77337164e960144"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.453016 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "e3109952d147441fa6f70bd8b431d958"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6fda72fca7e4c6d20cbc58e357d1286b"
server_key_iv: "ecb37e6951461ed0d77337164e960144"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.454437 7976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:15.455377 7991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.455641 7976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:15.455766 7976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "e3109952d147441fa6f70bd8b431d958"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6fda72fca7e4c6d20cbc58e357d1286b"
server_key_iv: "ecb37e6951461ed0d77337164e960144"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.455854 7976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:15.471218 7976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:15.474047 7976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:15.474252 7976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:15.474542 7976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:15.474951 7976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:15.475019 7976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.475065 7976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:15.475093 7976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.481715 7976 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:45149
I20260808 22:55:15.481760 8124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:45149 every 8 connection(s)
I20260808 22:55:15.482151 7976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:15.485117 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 7976
I20260808 22:55:15.485270 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:15.485481 1872 external_mini_cluster.cc:1428] Setting key 45f058d68dceecf8269672c97dfb0241
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:15.494338 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.483039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:42405 (local address 127.1.212.62:40753)
0808 22:55:15.483129 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.483132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.483333 (+ 201us) server_negotiation.cc:407] Connection header received
0808 22:55:15.484324 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.484325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.484337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.484389 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.485135 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.485839 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.486877 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.487119 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.487197 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.489895 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.489897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.489898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.489908 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.492041 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.492686 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.492688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.492689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.492813 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.493329 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.493330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.493331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.493538 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.493618 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.494087 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.494215 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:55:15.494494 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.482999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40753 (local address 127.1.212.1:42405)
0808 22:55:15.483204 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:15.483245 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:15.484223 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:15.484454 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:15.484457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:15.484733 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:15.485036 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.485042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.485929 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.485930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.486777 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.486780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.487223 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.487224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.487360 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.488005 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:15.488015 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:15.489742 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:15.492159 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.492170 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.492173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.492571 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.492922 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.492923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.492923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.493212 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.493619 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:15.493627 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:15.493851 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:55:15.493996 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 22:55:15.494212 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260808 22:55:15.495209 8125 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40753
I20260808 22:55:15.495362 8125 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:15.495604 8125 heartbeater.cc:511] Master 127.1.212.62:40753 requested a full tablet report, sending...
I20260808 22:55:15.496351 7908 ts_manager.cc:194] Registered new tserver with Master: e3109952d147441fa6f70bd8b431d958 (127.1.212.1:45149)
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
I20260808 22:55:15.497189 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:42405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:15.518385 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:15.612481 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:15.612792 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:15.612869 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:15.614223 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:15.614318 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:15.614372 8130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:15.616143 8130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:15.616468 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:15.616758 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:55:15.622383 8130 server_base.cc:1034] running on GCE node
W20260808 22:55:15.622272 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.622341 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.622272 8138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:15.622788 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:15.623107 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:15.624334 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786229715624281 us; error 81 us; skew 500 ppm
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.627975 8130 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:15.628540 8130 webserver.cc:460] Webserver started at http://127.1.212.2:36273/ using document root <none> and password file <none>
I20260808 22:55:15.628885 8130 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:15.628968 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:15.629139 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:15.630314 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "5860babe193e4591853d70bf0ec95185"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6b19b7f791dac721c4c59312d631e85b"
server_key_iv: "90d92e8a26fd43e9255f9c16be12ad9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.630721 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "5860babe193e4591853d70bf0ec95185"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6b19b7f791dac721c4c59312d631e85b"
server_key_iv: "90d92e8a26fd43e9255f9c16be12ad9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.632216 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:15.633241 8145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.633438 8130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:15.633558 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "5860babe193e4591853d70bf0ec95185"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "6b19b7f791dac721c4c59312d631e85b"
server_key_iv: "90d92e8a26fd43e9255f9c16be12ad9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.633641 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:15.654837 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:15.657419 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:15.657634 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:15.657938 8130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:15.658322 8130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:15.658385 8130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.658437 8130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:15.658465 8130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.665558 8130 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:43971
I20260808 22:55:15.665619 8268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:43971 every 8 connection(s)
I20260808 22:55:15.666052 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:15.676100 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8130
I20260808 22:55:15.676222 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:15.676486 1872 external_mini_cluster.cc:1428] Setting key 41339dddbbf0ed0beeefb938fc1bc271
I20260808 22:55:15.677498 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.667000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:49233 (local address 127.1.212.62:40753)
0808 22:55:15.667053 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.667055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.667332 (+ 277us) server_negotiation.cc:407] Connection header received
0808 22:55:15.668520 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.668522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.668539 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.668606 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.669492 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.670073 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.671249 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.671476 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.671568 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.674176 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.674178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.674179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.674187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.675713 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.676359 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.676361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.676361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.676422 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.676771 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.676797 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.676797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.676944 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.677031 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.677340 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.677423 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:55:15.677830 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.666982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40753 (local address 127.1.212.2:49233)
0808 22:55:15.667190 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:15.667232 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:15.668365 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:15.668703 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:15.668707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:15.669122 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:15.669364 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.669369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.670194 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.670195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.671138 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.671140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.671583 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.671584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.671740 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.672332 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:15.672342 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:15.674062 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:15.675820 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.675828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.675831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.676267 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.676498 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.676499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.676499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.676667 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.677021 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:15.677029 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:15.677187 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 22:55:15.677322 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 22:55:15.677541 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260808 22:55:15.678360 8269 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40753
I20260808 22:55:15.678479 8269 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:15.678716 8269 heartbeater.cc:511] Master 127.1.212.62:40753 requested a full tablet report, sending...
I20260808 22:55:15.679190 7908 ts_manager.cc:194] Registered new tserver with Master: 5860babe193e4591853d70bf0ec95185 (127.1.212.2:43971)
I20260808 22:55:15.679628 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:49233
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:15.709051 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:15.796692 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:15.796972 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:15.796998 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:15.798241 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:15.798291 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:15.798321 8274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:15.800199 8274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:15.800441 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:15.800647 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:15.806283 8279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.806291 8282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:15.806401 8280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:15.806481 8274 server_base.cc:1034] running on GCE node
I20260808 22:55:15.806807 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:15.807113 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:15.808322 8274 hybrid_clock.cc:648] HybridClock initialized: now 1786229715808251 us; error 82 us; skew 500 ppm
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.811466 8274 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:15.812016 8274 webserver.cc:460] Webserver started at http://127.1.212.3:36911/ using document root <none> and password file <none>
I20260808 22:55:15.812294 8274 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:15.812371 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:15.812536 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:15.813784 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "7fd8ba62bb634fcc8676e53a9b69ddda"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "ec39d0bb17390537237dab04862a4db6"
server_key_iv: "17cd1114afd9c5fdbd081b9423e52beb"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.814189 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "7fd8ba62bb634fcc8676e53a9b69ddda"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "ec39d0bb17390537237dab04862a4db6"
server_key_iv: "17cd1114afd9c5fdbd081b9423e52beb"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.815841 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:15.816761 8289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.817042 8274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:15.817160 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "7fd8ba62bb634fcc8676e53a9b69ddda"
format_stamp: "Formatted at 2026-08-08 22:55:15 on dist-test-slave-1x4g"
server_key: "ec39d0bb17390537237dab04862a4db6"
server_key_iv: "17cd1114afd9c5fdbd081b9423e52beb"
server_key_version: "encryptionkey@0"
I20260808 22:55:15.817255 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:15.826961 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:15.829600 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:15.829725 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:15.829936 8274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:15.830291 8274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:15.830331 8274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.830355 8274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:15.830404 8274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:15.836762 8274 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:38067
I20260808 22:55:15.836862 8416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:38067 every 8 connection(s)
I20260808 22:55:15.837256 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:15.845657 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8274
I20260808 22:55:15.845777 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:15.845978 1872 external_mini_cluster.cc:1428] Setting key c613fa913d132f1d0957812eac00679c
I20260808 22:55:15.847783 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.838187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:47001 (local address 127.1.212.62:40753)
0808 22:55:15.838282 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.838285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.838472 (+ 187us) server_negotiation.cc:407] Connection header received
0808 22:55:15.839486 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.839487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.839500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.839579 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.840228 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.840731 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.841935 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.842150 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.842215 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.844515 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.844517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.844517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.844526 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.846206 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.846798 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.846799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.846799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.846873 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.847190 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.847191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.847191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.847313 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.847368 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.847644 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.847716 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:55:15.848047 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.838139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40753 (local address 127.1.212.3:47001)
0808 22:55:15.838343 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:15.838385 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:15.839354 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:15.839641 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:15.839644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:15.839928 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:15.840146 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.840150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.840943 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.840944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.841814 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:15.841817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.842239 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.842240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:15.842361 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.842895 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:15.842904 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:15.844395 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:15.846301 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.846314 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.846317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.846710 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.846948 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:15.846949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:15.846950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:15.847120 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:15.847373 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:15.847378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:15.847526 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 22:55:15.847617 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:15.847758 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
I20260808 22:55:15.848541 8417 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40753
I20260808 22:55:15.848649 8417 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:15.848863 8417 heartbeater.cc:511] Master 127.1.212.62:40753 requested a full tablet report, sending...
I20260808 22:55:15.849288 7908 ts_manager.cc:194] Registered new tserver with Master: 7fd8ba62bb634fcc8676e53a9b69ddda (127.1.212.3:38067)
I20260808 22:55:15.849674 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:47001
I20260808 22:55:15.858109 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/test-admin.keytab.
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.877676 1872 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 22:55:15.879527 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:15.878682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.1.212.1:45149)
0808 22:55:15.878873 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.878878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.878888 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:15.878917 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.878919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.878935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.879010 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.879414 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38234: BlockingRecv error: recv got EOF from 127.0.0.1:38234 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
W20260808 22:55:15.879577 1872 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.880697 1872 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.881793 1872 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:55:15.890041 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.882452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.1.212.1:45149)
0808 22:55:15.882504 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.882506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.882540 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:55:15.882599 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.882599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.882610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.882662 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.883050 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.883799 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.884572 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.884978 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.885041 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.887363 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.887374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.887374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.887382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.888927 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.889273 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.889276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.889276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.889335 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.889506 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.889506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.889507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.889648 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.889768 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.889825 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.889893 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 22:55:15.892181 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:15.891587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.1.212.1:45149)
0808 22:55:15.891653 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.891655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.891692 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:55:15.891760 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.891761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.891772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.891816 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.892129 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38246: BlockingRecv error: recv got EOF from 127.0.0.1:38246 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 22:55:15.892277 1872 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.893400 1872 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.894289 1872 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:55:15.902395 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.894903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.1.212.1:45149)
0808 22:55:15.894979 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.894981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.895061 (+ 80us) server_negotiation.cc:407] Connection header received
0808 22:55:15.895153 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.895154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.895177 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.895242 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.895667 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.896187 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.897006 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.897220 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.897331 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.899545 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.899547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.899548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.899556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.901181 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.901564 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.901565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.901565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.901630 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.901845 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.901846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.901846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.901965 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.902081 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.902247 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.902322 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 22:55:15.904367 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:15.903544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.1.212.1:45149)
0808 22:55:15.903587 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.903589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.903677 (+ 88us) server_negotiation.cc:407] Connection header received
0808 22:55:15.903730 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.903731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.903739 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.903785 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.904291 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38264: BlockingRecv error: recv got EOF from 127.0.0.1:38264 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 22:55:15.904403 1872 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229688278322-1872-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.905557 1872 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:15.906519 1872 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:55:15 dist-test-slave-1x4g krb5kdc[7857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.1@KRBTEST.COM
I20260808 22:55:15.914000 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:15.907200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.1.212.1:45149)
0808 22:55:15.907266 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:15.907268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:15.907314 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:55:15.907378 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:15.907379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:15.907388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:15.907430 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:15.907794 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.908273 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.909076 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:15.909262 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:15.909318 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:15.911407 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:15.911409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:15.911409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:15.911418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:15.912932 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.913279 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.913280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.913280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.913339 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:15.913526 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:15.913527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:15.913527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:15.913633 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:15.913708 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:15.913805 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 22:55:15.913869 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:55:15.914752 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7976
I20260808 22:55:15.922086 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8130
I20260808 22:55:15.929158 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8274
I20260808 22:55:15.935384 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 7871
2026-08-08T22:55:15Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2018 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:16 dist-test-slave-1x4g krb5kdc[8442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:16 dist-test-slave-1x4g krb5kdc[8442](info): set up 2 sockets
Aug 08 22:55:16 dist-test-slave-1x4g krb5kdc[8442](info): commencing operation
krb5kdc: starting...
W20260808 22:55:17.026521 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab.
Entry for principal oryx/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab.
I20260808 22:55:17.164348 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34949
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab
--principal=oryx/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:17.250352 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:17.250635 8457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:17.250679 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:17.251937 8457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:17.252022 8457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:17.252048 8457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:17.252070 8457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:17.252101 8457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:17.253759 8457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab
--principal=oryx/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:17.254099 8457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:17.254357 8457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:17.259867 8467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.259963 8457 server_base.cc:1034] running on GCE node
W20260808 22:55:17.259877 8465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.259867 8463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.260521 8457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:17.260869 8457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:17.262079 8457 hybrid_clock.cc:648] HybridClock initialized: now 1786229717262047 us; error 44 us; skew 500 ppm
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:17.264952 8457 init.cc:377] Logged in from keytab as oryx/127.1.212.62@KRBTEST.COM (short username oryx)
I20260808 22:55:17.265496 8457 webserver.cc:460] Webserver started at http://127.1.212.62:41733/ using document root <none> and password file <none>
I20260808 22:55:17.265761 8457 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:17.265832 8457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:17.265997 8457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:17.267122 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f4620e1a99a84be798495865f881eea1"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "eaf3f11fc8a2b949652542b5e04a9017"
server_key_iv: "d692b0228792231c936a938210e870a0"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.267515 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f4620e1a99a84be798495865f881eea1"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "eaf3f11fc8a2b949652542b5e04a9017"
server_key_iv: "d692b0228792231c936a938210e870a0"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.268982 8457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 22:55:17.269843 8475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.270057 8457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:17.270183 8457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f4620e1a99a84be798495865f881eea1"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "eaf3f11fc8a2b949652542b5e04a9017"
server_key_iv: "d692b0228792231c936a938210e870a0"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.270267 8457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:17.288941 8457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:17.291554 8457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:17.291739 8457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:17.296316 8457 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:34949
I20260808 22:55:17.296356 8535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:34949 every 8 connection(s)
I20260808 22:55:17.296846 8457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:17.297935 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:17.300462 8536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Bootstrap starting.
I20260808 22:55:17.300925 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8457
I20260808 22:55:17.301056 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:17.301268 1872 external_mini_cluster.cc:1428] Setting key c0d9db35e28893634f0f689fca60ba3d
I20260808 22:55:17.301327 8536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:17.301697 8536 log.cc:824] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:17.302663 8536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: No bootstrap required, opened a new log
I20260808 22:55:17.305248 8536 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } }
I20260808 22:55:17.305461 8536 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:17.305495 8536 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4620e1a99a84be798495865f881eea1, State: Initialized, Role: FOLLOWER
I20260808 22:55:17.305658 8536 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } }
I20260808 22:55:17.305785 8536 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:17.305871 8536 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:17.305931 8536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:17.306756 8536 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } }
I20260808 22:55:17.306875 8536 leader_election.cc:302] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4620e1a99a84be798495865f881eea1; no voters:
I20260808 22:55:17.307062 8536 leader_election.cc:288] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:17.307153 8543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:17.307423 8543 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 LEADER]: Becoming Leader. State: Replica: f4620e1a99a84be798495865f881eea1, State: Running, Role: LEADER
I20260808 22:55:17.307451 8536 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:17.307564 8543 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } }
I20260808 22:55:17.308079 8543 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4620e1a99a84be798495865f881eea1. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } }
I20260808 22:55:17.308091 8544 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } }
I20260808 22:55:17.308270 8544 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:17.308538 8543 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:17.308549 8552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:17.309283 8552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:17.313557 8552 catalog_manager.cc:1347] Generated new cluster ID: 62a5e3f51b3f40529bb6c28afe43a13c
I20260808 22:55:17.313621 8552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:17.314369 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.301889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.1.212.62:34949)
0808 22:55:17.302127 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:17.302136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:17.302164 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:55:17.302841 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:17.302852 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:17.302925 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:17.303094 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:17.303750 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.305296 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.306343 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.307026 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.307165 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.310366 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:17.310370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:17.310371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:17.310387 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:17.312292 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.312769 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.312814 (+ 45us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.312814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.312925 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.313344 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.313345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.313346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.313700 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:17.313871 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:17.313998 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 22:55:17.314146 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":146,"threads_started":1}
I20260808 22:55:17.319092 8552 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:17.319656 8552 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:17.327427 8552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Generated new TSK 0
I20260808 22:55:17.327684 8552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
I20260808 22:55:17.365091 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:17.453289 8564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:17.453552 8564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:17.453603 8564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:17.455055 8564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:17.455152 8564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:17.455206 8564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:17.457065 8564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:17.457365 8564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:17.457626 8564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:17.463248 8571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.463236 8570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.463238 8573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.463357 8564 server_base.cc:1034] running on GCE node
I20260808 22:55:17.463739 8564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:17.464031 8564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:17.465217 8564 hybrid_clock.cc:648] HybridClock initialized: now 1786229717465194 us; error 36 us; skew 500 ppm
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:17.468076 8564 init.cc:377] Logged in from keytab as oryx/127.1.212.1@KRBTEST.COM (short username oryx)
I20260808 22:55:17.468616 8564 webserver.cc:460] Webserver started at http://127.1.212.1:35887/ using document root <none> and password file <none>
I20260808 22:55:17.468912 8564 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:17.468964 8564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:17.469069 8564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:17.470161 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "d7f6252b13bd4349b9cde0e74bcffcd8"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "9f3b76d6b9c2fcdcf58a603b94293976"
server_key_iv: "bb284480436030e2cf9743b7912cf11e"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.470477 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "d7f6252b13bd4349b9cde0e74bcffcd8"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "9f3b76d6b9c2fcdcf58a603b94293976"
server_key_iv: "bb284480436030e2cf9743b7912cf11e"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.471877 8564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:17.472676 8583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.472924 8564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:17.473076 8564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "d7f6252b13bd4349b9cde0e74bcffcd8"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "9f3b76d6b9c2fcdcf58a603b94293976"
server_key_iv: "bb284480436030e2cf9743b7912cf11e"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.473188 8564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:17.486194 8564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:17.489063 8564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:17.489259 8564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:17.489533 8564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:17.489885 8564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:17.489941 8564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.489977 8564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:17.490028 8564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.496381 8564 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:35089
I20260808 22:55:17.496433 8708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:35089 every 8 connection(s)
I20260808 22:55:17.496879 8564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:17.502022 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8564
I20260808 22:55:17.502156 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:17.502380 1872 external_mini_cluster.cc:1428] Setting key b5115cfc93e8d6f6dfa04a11be03135c
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:17.508623 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.497850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:45321 (local address 127.1.212.62:34949)
0808 22:55:17.497921 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:17.497923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:17.498235 (+ 312us) server_negotiation.cc:407] Connection header received
0808 22:55:17.498660 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:17.498661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:17.498673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:17.498746 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:17.499787 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.500330 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.501684 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.501921 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.501990 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.504978 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:17.504979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:17.504980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:17.504989 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:17.506923 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.507427 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.507428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.507429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.507497 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.507860 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.507861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.507862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.508001 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:17.508088 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:17.508408 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 22:55:17.508512 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:55:17.508898 8712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.497805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:45321)
0808 22:55:17.498016 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:17.498062 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:17.498570 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:17.498832 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:17.498836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:17.499464 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:17.499676 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.499681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.500461 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.500462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.501566 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.501570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.502050 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.502051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.502238 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.502974 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:17.502984 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:17.504863 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:17.507051 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.507062 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.507065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.507335 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.507572 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.507573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.507573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.507786 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.508087 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:17.508097 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:17.508259 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 22:55:17.508374 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:55:17.508571 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
I20260808 22:55:17.509431 8710 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:17.509562 8710 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:17.509778 8710 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:17.510371 8493 ts_manager.cc:194] Registered new tserver with Master: d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1:35089)
I20260808 22:55:17.511045 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:45321
WARNING: no policy specified for oryx/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
I20260808 22:55:17.536199 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:17.626258 8715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:17.626505 8715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:17.626533 8715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:17.627713 8715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:17.627756 8715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:17.627784 8715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:17.629820 8715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:17.630065 8715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:17.630270 8715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:17.635627 8721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.635624 8723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.635818 8725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.635988 8715 server_base.cc:1034] running on GCE node
I20260808 22:55:17.636291 8715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:17.636617 8715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:17.637794 8715 hybrid_clock.cc:648] HybridClock initialized: now 1786229717637785 us; error 39 us; skew 500 ppm
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:17.640648 8715 init.cc:377] Logged in from keytab as oryx/127.1.212.2@KRBTEST.COM (short username oryx)
I20260808 22:55:17.641247 8715 webserver.cc:460] Webserver started at http://127.1.212.2:46463/ using document root <none> and password file <none>
I20260808 22:55:17.641512 8715 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:17.641582 8715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:17.641741 8715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:17.642871 8715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "d570333d395c4b39a63cbd700fdd6f50"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "1d6fe902e27c9b48c181f1f3298177ab"
server_key_iv: "26a938ffef92f6a67623f2c36cc14ed6"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.643266 8715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "d570333d395c4b39a63cbd700fdd6f50"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "1d6fe902e27c9b48c181f1f3298177ab"
server_key_iv: "26a938ffef92f6a67623f2c36cc14ed6"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.644739 8715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:17.645677 8733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.645980 8715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:17.646092 8715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "d570333d395c4b39a63cbd700fdd6f50"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "1d6fe902e27c9b48c181f1f3298177ab"
server_key_iv: "26a938ffef92f6a67623f2c36cc14ed6"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.646173 8715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:17.660501 8715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:17.662858 8715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:17.663002 8715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:17.663203 8715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:17.663481 8715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:17.663513 8715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.663537 8715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:17.663549 8715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.669648 8715 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44937
I20260808 22:55:17.669746 8858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44937 every 8 connection(s)
I20260808 22:55:17.670076 8715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:17.672945 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8715
I20260808 22:55:17.673101 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:17.673295 1872 external_mini_cluster.cc:1428] Setting key 3745c328c856b162ebabdbd903ab5d81
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:17.681212 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.670866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:43243 (local address 127.1.212.62:34949)
0808 22:55:17.670928 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:17.670930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:17.671115 (+ 185us) server_negotiation.cc:407] Connection header received
0808 22:55:17.671697 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:17.671699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:17.671712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:17.671774 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:17.672887 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.673556 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.674766 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.675098 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.675182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.677720 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:17.677722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:17.677723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:17.677733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:17.679544 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.679998 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.680000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.680000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.680072 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.680347 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.680348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.680348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.680533 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:17.680618 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:17.680950 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 22:55:17.681061 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:55:17.681244 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.670814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:43243)
0808 22:55:17.670976 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:17.671025 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:17.671576 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:17.671847 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:17.671851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:17.672476 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:17.672755 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.672762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.673674 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.673675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.674669 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.674672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.675250 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.675252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.675382 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.676051 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:17.676061 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:17.677615 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:17.679650 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.679659 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.679661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.679907 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.680155 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.680156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.680157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.680271 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.680609 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:17.680615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:17.680750 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 22:55:17.680898 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 22:55:17.681062 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260808 22:55:17.681782 8859 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:17.681918 8859 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:17.682159 8859 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:17.682601 8493 ts_manager.cc:194] Registered new tserver with Master: d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:17.683001 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:43243
WARNING: no policy specified for oryx/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
I20260808 22:55:17.704864 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:17.792795 8865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:17.793104 8865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:17.793135 8865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:17.794442 8865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:17.794489 8865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:17.794518 8865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:17.796049 8865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:17.796264 8865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:17.796460 8865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:17.801924 8870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.801924 8873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.801934 8871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.802222 8865 server_base.cc:1034] running on GCE node
I20260808 22:55:17.802457 8865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:17.802718 8865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:17.803895 8865 hybrid_clock.cc:648] HybridClock initialized: now 1786229717803880 us; error 31 us; skew 500 ppm
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:17.806726 8865 init.cc:377] Logged in from keytab as oryx/127.1.212.3@KRBTEST.COM (short username oryx)
I20260808 22:55:17.807299 8865 webserver.cc:460] Webserver started at http://127.1.212.3:36169/ using document root <none> and password file <none>
I20260808 22:55:17.807621 8865 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:17.807693 8865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:17.807842 8865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:17.809191 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "0c6b3632fbda4fc9bdacc093b913f782"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "e549ff28c89222f9c69ce49b7e093bf6"
server_key_iv: "e5ee558fd459dee3c6423e54a9e4b1c9"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.809607 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "0c6b3632fbda4fc9bdacc093b913f782"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "e549ff28c89222f9c69ce49b7e093bf6"
server_key_iv: "e5ee558fd459dee3c6423e54a9e4b1c9"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.811108 8865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:17.811957 8881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.812243 8865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:17.812377 8865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "0c6b3632fbda4fc9bdacc093b913f782"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "e549ff28c89222f9c69ce49b7e093bf6"
server_key_iv: "e5ee558fd459dee3c6423e54a9e4b1c9"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.812472 8865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:17.824533 8865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:17.826830 8865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:17.827049 8865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:17.827327 8865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:17.827754 8865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:17.827792 8865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.827817 8865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:17.827829 8865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.834971 8865 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:36953
I20260808 22:55:17.835011 9010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:36953 every 8 connection(s)
I20260808 22:55:17.835382 8865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:17.842024 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 8865
I20260808 22:55:17.842141 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:17.842336 1872 external_mini_cluster.cc:1428] Setting key cf63d502e2b808d3ecb6ceb1542311dc
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:17.847368 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.836169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:49769 (local address 127.1.212.62:34949)
0808 22:55:17.836218 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:17.836220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:17.836462 (+ 242us) server_negotiation.cc:407] Connection header received
0808 22:55:17.837049 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:17.837050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:17.837062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:17.837149 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:17.838220 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.838746 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.839875 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.840096 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.840176 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.843943 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:17.843944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:17.843945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:17.843955 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:17.845635 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.846117 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.846118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.846118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.846182 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:17.846464 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:17.846466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:17.846466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:17.846606 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:17.846689 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:17.847097 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 22:55:17.847292 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:55:17.847384 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:17.836133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:49769)
0808 22:55:17.836305 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:17.836352 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:17.836952 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:17.837230 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:17.837234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:17.837855 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:17.838085 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.838091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.838877 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.838879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.839766 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:17.839769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:17.840301 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:17.840303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:17.840520 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:17.841825 (+ 1305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:17.841841 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:17.843817 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:17.845746 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.845757 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.845759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.846034 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.846251 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:17.846252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:17.846253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:17.846378 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:17.846741 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:17.846749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:17.846916 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 22:55:17.847026 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:55:17.847189 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260808 22:55:17.848004 9011 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:17.848130 9011 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:17.848366 9011 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:17.848878 8493 ts_manager.cc:194] Registered new tserver with Master: 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:17.849332 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:49769
I20260808 22:55:17.855149 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.212.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.61.keytab.
Entry for principal oryx/127.1.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.61.keytab.
I20260808 22:55:17.889083 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:39075
--webserver_interface=127.1.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.61.keytab
--principal=oryx/127.1.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34949,127.1.212.61:39075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:17.977758 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:17.978017 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:17.978044 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:17.979280 9018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:17.979323 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:17.979338 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:17.979351 9018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:17.979363 9018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:17.980934 9018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34949,127.1.212.61:39075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.61.keytab
--principal=oryx/127.1.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:39075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
W20260808 22:55:17.981062 9018 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.212.62:34949,127.1.212.61:39075'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 22:55:17.981189 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:17.981384 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:17.986426 9024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.986471 9023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:17.986518 9026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:17.986871 9018 server_base.cc:1034] running on GCE node
I20260808 22:55:17.987160 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:17.987540 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:17.988714 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786229717988695 us; error 41 us; skew 500 ppm
Aug 08 22:55:17 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:17.991524 9018 init.cc:377] Logged in from keytab as oryx/127.1.212.61@KRBTEST.COM (short username oryx)
I20260808 22:55:17.992074 9018 webserver.cc:460] Webserver started at http://127.1.212.61:37689/ using document root <none> and password file <none>
I20260808 22:55:17.992323 9018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:17.992393 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:17.992558 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:17.993788 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/instance:
uuid: "5afb3209a4534ec88b758e874b74fbbd"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "0accb54aa15f3d7ec9cc243139399838"
server_key_iv: "282f49a4e6b1cff390fd834dea8dcdf1"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.994268 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal/instance:
uuid: "5afb3209a4534ec88b758e874b74fbbd"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "0accb54aa15f3d7ec9cc243139399838"
server_key_iv: "282f49a4e6b1cff390fd834dea8dcdf1"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.996229 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:55:17.997529 9037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:17.997694 9018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:17.997789 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "5afb3209a4534ec88b758e874b74fbbd"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "0accb54aa15f3d7ec9cc243139399838"
server_key_iv: "282f49a4e6b1cff390fd834dea8dcdf1"
server_key_version: "encryptionkey@0"
I20260808 22:55:17.997865 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.019043 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.021087 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:18.021219 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.025807 9018 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.61:39075
I20260808 22:55:18.025843 9095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.61:39075 every 8 connection(s)
I20260808 22:55:18.026254 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
I20260808 22:55:18.027369 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:18.029098 9096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.61@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:18.035746 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 9018
I20260808 22:55:18.039740 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.029944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.1.212.62:34949)
0808 22:55:18.030012 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.030014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.030291 (+ 277us) server_negotiation.cc:407] Connection header received
0808 22:55:18.030758 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.030759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.030771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.030827 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.031523 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.032037 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.033167 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.033362 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.033426 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.036044 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.036046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.036047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.036071 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.038008 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.038492 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.038494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.038494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.038569 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.038904 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.038904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.038905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.039085 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.039146 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.039544 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.039642 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:18.039885 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.029902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54262)
0808 22:55:18.030142 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.030181 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.030651 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.030911 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.030915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.031202 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.031418 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.031422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.032146 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.032148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.033084 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.033087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.033494 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.033495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.033654 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.034248 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.034257 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.035930 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.038132 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.038140 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.038142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.038392 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.038651 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.038652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.038653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.038822 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.039164 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.039171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.039341 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.039482 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.039662 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":129,"threads_started":1}
I20260808 22:55:18.040565 9096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.61@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.045971 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.036383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.1.212.61:39075)
0808 22:55:18.036702 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.036707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.036734 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:55:18.036759 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.036761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.036796 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.036874 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.037483 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.038476 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.039527 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.040179 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.040269 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.042834 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.042845 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.042845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.042853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.044580 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.045035 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.045038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.045038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.045098 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.045327 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.045328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.045328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.045491 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.045679 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.045754 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.045838 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":110,"threads_started":1}
I20260808 22:55:18.048415 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.040988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48112)
0808 22:55:18.041239 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.041250 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.041304 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.041448 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.041448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.041598 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.041643 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.041645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.042323 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.042325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.042876 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.042878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.043327 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.043328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.043396 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.043785 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.043790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.045064 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.047387 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.047388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.047389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.047557 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.047768 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.047769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.047770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.047825 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.048082 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.048083 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.048122 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.048190 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.048297 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260808 22:55:18.048516 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.041072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.1.212.61:39075)
0808 22:55:18.041250 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.041253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.041287 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:55:18.041359 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.041360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.041370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.041426 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.041726 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.042240 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.042987 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.043246 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.043330 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.045146 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.045148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.045148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.045155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.047289 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.047642 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.047643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.047644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.047701 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.047898 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.047899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.047899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.048011 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.048091 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.048216 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.048349 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":159,"thread_start_us":130,"threads_started":1}
I20260808 22:55:18.050065 9096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Bootstrap starting.
I20260808 22:55:18.050740 9096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:18.050987 9096 log.cc:824] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:18.051695 9096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: No bootstrap required, opened a new log
I20260808 22:55:18.053205 9096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } }
I20260808 22:55:18.053337 9096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:18.053359 9096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5afb3209a4534ec88b758e874b74fbbd, State: Initialized, Role: FOLLOWER
I20260808 22:55:18.053491 9096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } }
I20260808 22:55:18.053790 9096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:18.053782 9103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } } }
I20260808 22:55:18.053949 9103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:55:18.055722 9117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:18.055805 9117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:18.057727 9018 master_runner.cc:428] Detected that this master 5afb3209a4534ec88b758e874b74fbbd is joining an existing cluster
I20260808 22:55:18.057794 9018 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.212.61:39075
I20260808 22:55:18.066110 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.058941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.1.212.61:39075)
0808 22:55:18.058986 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.058988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.059385 (+ 397us) server_negotiation.cc:407] Connection header received
0808 22:55:18.059472 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.059473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.059482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.059528 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.059979 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.060716 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.061551 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.061787 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.061876 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.063038 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.063039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.063040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.063047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.064687 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065128 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.065129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.065130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.065182 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065526 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.065527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.065527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.065637 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.065716 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.065842 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.065997 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":24}
I20260808 22:55:18.066192 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.058979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48120)
0808 22:55:18.059335 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.059343 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.059404 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.059543 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.059544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.059826 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.059895 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.059896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.060836 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.060836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.061452 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.061454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.061894 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.061895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.061955 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.062462 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.062468 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.062934 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.064818 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.064819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.064820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.065062 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.065256 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.065257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.065316 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.065707 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.065707 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.065748 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.065817 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.066053 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":160,"thread_start_us":104,"threads_started":1}
I20260808 22:55:18.066663 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.058970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.1.212.62:34949)
0808 22:55:18.059022 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.059024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.059222 (+ 198us) server_negotiation.cc:407] Connection header received
0808 22:55:18.059302 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.059303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.059318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.059380 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.059895 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.060707 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.061917 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.062235 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.062280 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.063339 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.063341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.063342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.063351 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.064980 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065396 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.065397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.065398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.065453 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065721 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.065722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.065722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.065870 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.065916 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.066129 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.066557 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:55:18.067018 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.058906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54276)
0808 22:55:18.059142 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.059153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.059238 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.059410 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.059412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.059722 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.059811 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.059813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.060826 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.060827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.061798 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.061801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.062365 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.062365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.062433 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.062853 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.062859 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.063251 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.065079 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.065082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.065314 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.065536 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.065537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.065537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.065632 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.065969 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.065970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.066014 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.066798 (+ 784us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.066915 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":171,"threads_started":1}
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.62@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.077082 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.069475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.1.212.61:39075)
0808 22:55:18.069517 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.069518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.069689 (+ 171us) server_negotiation.cc:407] Connection header received
0808 22:55:18.069787 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.069788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.069798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.069855 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.070298 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.070766 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.071545 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.071752 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.071838 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.074247 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.074248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.074249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.074257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.075675 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.076098 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.076100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.076100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.076163 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.076438 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.076439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.076439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.076563 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.076659 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.076893 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.076979 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:55:18.077114 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.069402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48130)
0808 22:55:18.069580 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.069625 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.069717 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.069862 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.069865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.070159 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.070212 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.070214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.070856 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.070857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.071457 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.071458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.071841 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.071842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.071924 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.072413 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.072418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.074142 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.075784 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.075793 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.075796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.076017 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.076248 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.076248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.076249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.076317 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.076649 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.076658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.076715 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.076860 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.076999 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260808 22:55:18.077894 8493 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.212.61:39075: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd"
member_type: NON_VOTER
last_known_addr {
host: "127.1.212.61"
port: 39075
}
attrs {
promote: true
}
}
dest_uuid: "f4620e1a99a84be798495865f881eea1"
cas_config_opid_index: -1
I20260808 22:55:18.078233 8493 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } }
I20260808 22:55:18.078738 9130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } }
I20260808 22:55:18.079123 9066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:18.079753 9130 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.212.61:39075
I20260808 22:55:18.079998 9129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } }
I20260808 22:55:18.080165 9066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 22:55:18.080202 9129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.080252 9018 master.cc:537] Master@127.1.212.61:39075 shutting down...
I20260808 22:55:18.080681 9129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:55:18.081547 8479 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 22:55:18.082079 8479 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 -> Peer 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61:39075): Couldn't send request to peer 5afb3209a4534ec88b758e874b74fbbd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:55:18.083312 9018 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:18.083411 9018 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:18.083474 9018 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: stopping tablet replica
I20260808 22:55:18.096663 9018 master.cc:552] Master@127.1.212.61:39075 shutdown complete.
I20260808 22:55:18.096740 9018 master_runner.cc:305] Clearing existing system tablet
I20260808 22:55:18.097002 9018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.097074 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.097867 9018 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 22:55:18.098423 9134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.098592 9018 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.098650 9018 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 22:55:18.098681 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "5afb3209a4534ec88b758e874b74fbbd"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "0accb54aa15f3d7ec9cc243139399838"
server_key_iv: "282f49a4e6b1cff390fd834dea8dcdf1"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.099427 9018 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:18.101226 9018 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:55:18.101339 9018 log.cc:1196] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 22:55:18.101570 9018 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Deleting consensus metadata
I20260808 22:55:18.101739 9018 master_runner.cc:315] Copying system tablet from 127.1.212.62:34949
I20260808 22:55:18.102490 9018 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: tablet copy: Beginning tablet copy session from remote peer at address 127.1.212.62:34949
I20260808 22:55:18.109717 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.102994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.1.212.62:34949)
0808 22:55:18.103057 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.103060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.103132 (+ 72us) server_negotiation.cc:407] Connection header received
0808 22:55:18.103232 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.103233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.103245 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.103326 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.103739 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.104273 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.105212 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.105423 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.105517 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.106585 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.106587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.106587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.106595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.108348 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.108843 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.108844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.108845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.108911 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.109112 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.109113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.109113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.109239 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.109336 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.109533 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.109618 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:55:18.109736 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.102913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54288)
0808 22:55:18.103054 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.103068 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.103154 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.103349 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.103350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.103567 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.103634 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.103637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.104389 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.104390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.105098 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.105101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.105521 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.105522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.105601 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.106134 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.106140 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.106465 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.108444 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.108445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.108446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.108694 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.108974 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.108975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.108975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.109045 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.109360 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.109361 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.109410 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.109501 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.109619 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260808 22:55:18.110373 8520 tablet_copy_service.cc:140] P f4620e1a99a84be798495865f881eea1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5afb3209a4534ec88b758e874b74fbbd ({username='oryx', principal='oryx/127.1.212.61@KRBTEST.COM'} at 127.0.0.1:54288)
I20260808 22:55:18.110500 8520 tablet_copy_service.cc:161] P f4620e1a99a84be798495865f881eea1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5afb3209a4534ec88b758e874b74fbbd at {username='oryx', principal='oryx/127.1.212.61@KRBTEST.COM'} at 127.0.0.1:54288: session id = 5afb3209a4534ec88b758e874b74fbbd-00000000000000000000000000000000
I20260808 22:55:18.111361 8520 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:55:18.112062 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:55:18.113220 9018 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: tablet copy: Starting download of 0 data blocks...
I20260808 22:55:18.113385 9018 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: tablet copy: Starting download of 1 WAL segments...
I20260808 22:55:18.114403 9018 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:55:18.115463 8520 tablet_copy_service.cc:342] P f4620e1a99a84be798495865f881eea1: Request end of tablet copy session 5afb3209a4534ec88b758e874b74fbbd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.212.61@KRBTEST.COM'} at 127.0.0.1:54288
I20260808 22:55:18.115540 8520 tablet_copy_service.cc:433] P f4620e1a99a84be798495865f881eea1: ending tablet copy session 5afb3209a4534ec88b758e874b74fbbd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5afb3209a4534ec88b758e874b74fbbd
I20260808 22:55:18.118582 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.119858 9141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.119896 9142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.125749 9144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.126104 9018 server_base.cc:1034] running on GCE node
I20260808 22:55:18.126253 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.126449 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.127585 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786229718127566 us; error 38 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.129274 9018 init.cc:377] Logged in from keytab as oryx/127.1.212.61@KRBTEST.COM (short username oryx)
I20260808 22:55:18.129619 9018 webserver.cc:460] Webserver started at http://127.1.212.61:44195/ using document root <none> and password file <none>
I20260808 22:55:18.129755 9018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.129801 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.130682 9018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.131407 9153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.131582 9018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.131644 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "5afb3209a4534ec88b758e874b74fbbd"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "0accb54aa15f3d7ec9cc243139399838"
server_key_iv: "282f49a4e6b1cff390fd834dea8dcdf1"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.131711 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.144724 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.146586 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.149590 9018 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.61:39075
I20260808 22:55:18.149613 9207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.61:39075 every 8 connection(s)
I20260808 22:55:18.149952 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
I20260808 22:55:18.150810 9208 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:55:18.151262 9208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Bootstrap starting.
I20260808 22:55:18.153980 9208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:55:18.154290 9208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: Bootstrap complete.
I20260808 22:55:18.154541 9208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } }
I20260808 22:55:18.154618 9208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5afb3209a4534ec88b758e874b74fbbd, State: Initialized, Role: LEARNER
I20260808 22:55:18.154646 9208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } }
I20260808 22:55:18.154886 9208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:18.154906 9211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } }
I20260808 22:55:18.155035 9211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: LEARNER
I20260808 22:55:18.156473 9224 catalog_manager.cc:1259] Loaded cluster ID: 62a5e3f51b3f40529bb6c28afe43a13c
I20260808 22:55:18.156682 9224 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: loading cluster ID for follower catalog manager: success
I20260808 22:55:18.157899 9224 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: acquiring CA information for follower catalog manager: success
I20260808 22:55:18.158277 9224 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:55:18.205976 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.198368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.1.212.61:39075)
0808 22:55:18.198534 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.198536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.198549 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:18.198581 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.198582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.198595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.198687 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.199244 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.200307 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.201222 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.201532 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.201619 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.202912 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.202913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.202914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.202923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.204533 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.204971 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.204972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.204973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.205044 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.205428 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.205429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.205429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.205577 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.205677 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.205772 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.205865 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.1.212.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.60.keytab.
Entry for principal oryx/127.1.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.60.keytab.
I20260808 22:55:18.236717 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:41701
--webserver_interface=127.1.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.60.keytab
--principal=oryx/127.1.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:18.326560 9228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:18.326790 9228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:18.326818 9228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:18.328071 9228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:18.328123 9228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:18.328137 9228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:18.328150 9228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:18.328162 9228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:18.329983 9228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.60.keytab
--principal=oryx/127.1.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:41701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:18.330334 9228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:18.330804 9228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.337146 9233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.337191 9236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.337152 9234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.337726 9228 server_base.cc:1034] running on GCE node
I20260808 22:55:18.337998 9228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.338330 9228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.339543 9228 hybrid_clock.cc:648] HybridClock initialized: now 1786229718339530 us; error 30 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.342449 9228 init.cc:377] Logged in from keytab as oryx/127.1.212.60@KRBTEST.COM (short username oryx)
I20260808 22:55:18.342993 9228 webserver.cc:460] Webserver started at http://127.1.212.60:42973/ using document root <none> and password file <none>
I20260808 22:55:18.343266 9228 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.343338 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.343497 9228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:18.344835 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/instance:
uuid: "0325606323134650a428f2ee67c851bc"
format_stamp: "Formatted at 2026-08-08 22:55:18 on dist-test-slave-1x4g"
server_key: "8e6a8a5ded95fbb24031bd9aab6a6438"
server_key_iv: "503dd6e763e7424a3a4e4cd9c5562e0c"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.345379 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal/instance:
uuid: "0325606323134650a428f2ee67c851bc"
format_stamp: "Formatted at 2026-08-08 22:55:18 on dist-test-slave-1x4g"
server_key: "8e6a8a5ded95fbb24031bd9aab6a6438"
server_key_iv: "503dd6e763e7424a3a4e4cd9c5562e0c"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.347018 9228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:55:18.347888 9244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.348109 9228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.348220 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "0325606323134650a428f2ee67c851bc"
format_stamp: "Formatted at 2026-08-08 22:55:18 on dist-test-slave-1x4g"
server_key: "8e6a8a5ded95fbb24031bd9aab6a6438"
server_key_iv: "503dd6e763e7424a3a4e4cd9c5562e0c"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.348306 9228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.374264 9228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.376508 9228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:18.376649 9228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.381690 9228 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.60:41701
I20260808 22:55:18.381788 9302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.60:41701 every 8 connection(s)
I20260808 22:55:18.382158 9228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
I20260808 22:55:18.383281 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:18.383579 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 9228
I20260808 22:55:18.385932 9303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.60@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:18.395155 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.386957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.1.212.62:34949)
0808 22:55:18.387010 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.387013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.387050 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:55:18.387150 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.387151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.387163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.387241 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.387749 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.388245 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.389333 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.389629 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.389752 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.392246 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.392248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.392248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.392258 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.393789 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.394198 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.394200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.394200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.394265 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.394519 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.394520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.394520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.394654 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.394729 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.394944 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.395037 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:55:18.395310 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.384294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.1.212.60:41701)
0808 22:55:18.384501 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.384509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.384539 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:55:18.385068 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.385078 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.385133 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.385299 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.385959 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.387224 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.387983 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.388420 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.388592 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.391428 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.391437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.391438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.391445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.393562 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.393991 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.394000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.394001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.394106 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.394332 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.394334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.394334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.394628 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.394822 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.394904 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.395069 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260808 22:55:18.395394 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.386656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54302)
0808 22:55:18.386934 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.386984 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.387084 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.387270 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.387273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.387577 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.387651 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.387653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.388364 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.388366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.389224 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.389227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.389771 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.389773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.389872 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.390406 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.390416 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.392148 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.393873 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.393882 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.393884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.394114 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.394340 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.394341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.394341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.394444 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.394734 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.394743 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.394816 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.394911 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.395242 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":168,"thread_start_us":105,"threads_started":1}
I20260808 22:55:18.396046 9303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.60@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.403501 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.396444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.1.212.61:39075)
0808 22:55:18.396510 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.396513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.396556 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:55:18.396652 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.396653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.396667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.396725 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.397063 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.397578 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.398301 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.398642 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.398730 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.400607 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.400609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.400609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.400618 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.402364 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.402764 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.402765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.402766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.402826 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.403028 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.403029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.403029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.403159 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.403214 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.403354 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.403434 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:55:18.403501 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.396375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48156)
0808 22:55:18.396509 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.396521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.396574 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.396729 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.396730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.396915 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.396974 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.396976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.397679 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.397681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.398206 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.398208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.398729 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.398730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.398821 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.399228 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.399232 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.400505 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.402480 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.402481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.402482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.402684 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.402904 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.402905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.402905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.402962 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.403222 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.403223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.403269 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.403329 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.403424 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 22:55:18.404147 9303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.60@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
I20260808 22:55:18.411330 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.404447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:56958)
0808 22:55:18.404513 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.404525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.404569 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.404682 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.404683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.404918 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.404955 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.404956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.405636 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.405637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.406113 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.406115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.406494 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.406502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.406553 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.407043 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.407048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.408376 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.410219 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.410220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.410220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.410398 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.410679 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.410680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.410680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.410738 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.411063 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.411064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.411107 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.411160 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.411257 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 22:55:18.411406 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.404499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56958 (local address 127.1.212.60:41701)
0808 22:55:18.404540 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.404542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.404553 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:18.404615 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.404616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.404625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.404678 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.405050 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.405541 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.406197 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.406408 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.406495 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.408524 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.408526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.408526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.408536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.410101 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.410541 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.410543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.410543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.410605 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.410836 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.410837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.410837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.410988 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.411067 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.411207 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.411316 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":24}
I20260808 22:55:18.413309 9303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Bootstrap starting.
I20260808 22:55:18.414312 9303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:18.414692 9303 log.cc:824] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:18.415585 9303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: No bootstrap required, opened a new log
I20260808 22:55:18.417482 9303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } }
I20260808 22:55:18.417699 9303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:18.417762 9303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0325606323134650a428f2ee67c851bc, State: Initialized, Role: FOLLOWER
I20260808 22:55:18.417953 9303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } }
I20260808 22:55:18.418340 9308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } } }
I20260808 22:55:18.418395 9303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:18.418464 9308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:55:18.420826 9319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:18.420907 9319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:18.423647 9228 master_runner.cc:428] Detected that this master 0325606323134650a428f2ee67c851bc is joining an existing cluster
I20260808 22:55:18.423707 9228 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.212.60:41701
I20260808 22:55:18.432803 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.425362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.1.212.61:39075)
0808 22:55:18.425408 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.425410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.425641 (+ 231us) server_negotiation.cc:407] Connection header received
0808 22:55:18.425755 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.425756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.425768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.425803 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.426323 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.427127 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428284 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428596 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428634 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.429891 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.429893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.429894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.429903 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.431504 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.431890 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.431891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.431892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.431956 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.432171 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.432172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.432172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.432304 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.432384 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.432554 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.432667 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:55:18.432829 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.425315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48172)
0808 22:55:18.425576 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.425586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.425671 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.425899 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.425899 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.426143 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.426227 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.426228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.427224 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.427225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428176 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.428180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428706 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428816 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.429374 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.429380 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.429774 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.431590 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.431591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.431592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.431808 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.432036 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.432037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.432037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.432121 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.432386 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.432387 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.432438 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.432526 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.432668 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":47,"thread_start_us":211,"threads_started":1}
I20260808 22:55:18.447211 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8564
I20260808 22:55:18.457032 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:35089
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:34949
--webserver_port=35887
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:55:18.458058 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.425695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.1.212.60:41701)
0808 22:55:18.425734 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.425735 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.426001 (+ 266us) server_negotiation.cc:407] Connection header received
0808 22:55:18.426096 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.426097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.426105 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.426132 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.426495 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.427177 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428278 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428565 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428606 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.454338 (+ 25732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.454340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.454340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.454349 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.456550 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.457070 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.457071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.457071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.457130 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.457356 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.457357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.457357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.457542 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.457590 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.457786 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.457915 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:55:18.458456 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.425767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:56970)
0808 22:55:18.425935 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.425944 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.426034 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.426185 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.426185 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.426370 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.426422 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.426424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.427252 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.427252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428176 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.428180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428706 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428819 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.453543 (+ 24724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.453551 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.454202 (+ 651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.456662 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.456663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.456664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.456983 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.457202 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.457202 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.457203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.457260 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.457632 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.457633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.457674 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.458190 (+ 516us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.458311 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":64,"thread_start_us":110,"threads_started":1}
I20260808 22:55:18.460415 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.424972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.1.212.62:34949)
0808 22:55:18.425082 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.425084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.425310 (+ 226us) server_negotiation.cc:407] Connection header received
0808 22:55:18.425438 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.425440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.425453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.425488 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.426057 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.426945 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428293 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428655 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428706 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.454369 (+ 25663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.454371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.454372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.454383 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.456869 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.459526 (+ 2657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.459529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.459530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.459629 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.459848 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.459849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.459849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.459991 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.460036 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.460267 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.460348 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 22:55:18.460444 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.424864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54308)
0808 22:55:18.425226 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.425241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.425345 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.425583 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.425584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.425841 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.425942 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.425945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.427067 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.427068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428168 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.428171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.428856 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.428857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.428934 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.453492 (+ 24558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.453502 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.454216 (+ 714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.456996 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.456998 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.456998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.459405 (+ 2407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.459713 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.459714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.459714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.459779 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.460086 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.460087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.460135 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.460207 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.460326 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":266,"threads_started":1}
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229717, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.62@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
I20260808 22:55:18.469835 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.462989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:56972)
0808 22:55:18.463075 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.463087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.463138 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.463275 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.463276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.463453 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.463498 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.463500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.464163 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.464164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.464699 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.464701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.465114 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.465116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.465189 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.465636 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.465642 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.466943 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.468623 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.468624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.468625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.468910 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.469121 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.469122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.469123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.469193 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.469479 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.469480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.469529 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.469615 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.469734 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 22:55:18.469835 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.463031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.1.212.60:41701)
0808 22:55:18.463073 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.463075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.463119 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:55:18.463203 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.463204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.463215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.463271 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.463578 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.464067 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.464817 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.465025 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.465103 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.467043 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.467045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.467045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.467053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.468515 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.468984 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.468986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.468986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.469051 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.469271 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.469272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.469272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.469401 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.469474 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.469644 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.469732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 22:55:18.470793 8493 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.212.60:41701: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0325606323134650a428f2ee67c851bc"
member_type: NON_VOTER
last_known_addr {
host: "127.1.212.60"
port: 41701
}
attrs {
promote: true
}
}
dest_uuid: "f4620e1a99a84be798495865f881eea1"
cas_config_opid_index: 5
I20260808 22:55:18.471115 8493 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } }
I20260808 22:55:18.471508 9130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0325606323134650a428f2ee67c851bc (127.1.212.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.472105 9276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:18.472592 9130 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.212.60:41701
I20260808 22:55:18.472667 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.472733 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.473469 9228 master.cc:537] Master@127.1.212.60:41701 shutting down...
I20260808 22:55:18.473507 9276 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 22:55:18.473798 9129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:18.474432 9308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:18.474537 9308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } }
I20260808 22:55:18.474818 9308 leader_election.cc:288] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949), 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61:39075)
I20260808 22:55:18.475178 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0325606323134650a428f2ee67c851bc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4620e1a99a84be798495865f881eea1" is_pre_election: true
I20260808 22:55:18.475304 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0325606323134650a428f2ee67c851bc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5afb3209a4534ec88b758e874b74fbbd" is_pre_election: true
I20260808 22:55:18.475412 9182 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Handling vote request from an unknown peer 0325606323134650a428f2ee67c851bc
I20260808 22:55:18.475468 9182 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 0325606323134650a428f2ee67c851bc for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
W20260808 22:55:18.475626 8479 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 -> Peer 0325606323134650a428f2ee67c851bc (127.1.212.60:41701): Couldn't send request to peer 0325606323134650a428f2ee67c851bc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:55:18.475808 9246 leader_election.cc:302] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0325606323134650a428f2ee67c851bc; no voters: 5afb3209a4534ec88b758e874b74fbbd, f4620e1a99a84be798495865f881eea1
I20260808 22:55:18.475950 9308 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:55:18.477802 9228 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:18.477903 9228 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:18.477948 9228 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: stopping tablet replica
I20260808 22:55:18.480492 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.471739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.1.212.61:39075)
0808 22:55:18.471785 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.471788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.475768 (+ 3980us) server_negotiation.cc:407] Connection header received
0808 22:55:18.475892 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.475893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.475915 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.475980 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:18.476242 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.477043 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.479253 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.479930 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.479980 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.480066 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.480354 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.480407 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:55:18.480746 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.471617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:48186)
0808 22:55:18.475708 (+ 4091us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.475721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.475800 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.476095 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.476096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.476098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:18.476151 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.476153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.477142 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.477144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.479144 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.479149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.480138 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.480139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.480226 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.480257 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.480629 (+ 372us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.480680 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4070}
I20260808 22:55:18.481130 9182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 22:55:18.481354 9130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 22:55:18.482026 9182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0325606323134650a428f2ee67c851bc (127.1.212.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.482316 9228 master.cc:552] Master@127.1.212.60:41701 shutdown complete.
I20260808 22:55:18.482411 9228 master_runner.cc:305] Clearing existing system tablet
I20260808 22:55:18.482586 9228 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.482661 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.483332 9211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4620e1a99a84be798495865f881eea1. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.483440 9211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: LEARNER
I20260808 22:55:18.483484 9211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: NON_VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: true } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.483521 9211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: LEARNER
I20260808 22:55:18.483572 9228 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.484058 9130 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: attempting to promote NON_VOTER 5afb3209a4534ec88b758e874b74fbbd to VOTER
I20260808 22:55:18.484233 9333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.484302 9130 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } }
I20260808 22:55:18.484402 9228 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.484448 9228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:18.484474 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "0325606323134650a428f2ee67c851bc"
format_stamp: "Formatted at 2026-08-08 22:55:18 on dist-test-slave-1x4g"
server_key: "8e6a8a5ded95fbb24031bd9aab6a6438"
server_key_iv: "503dd6e763e7424a3a4e4cd9c5562e0c"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.485108 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.484954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:56982)
0808 22:55:18.485028 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.485075 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 22:55:18.485280 9182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 LEARNER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 22:55:18.485545 9129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 22:55:18.485680 9228 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:18.485853 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.485731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:56992)
0808 22:55:18.485793 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.485821 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:55:18.486114 8479 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 -> Peer 0325606323134650a428f2ee67c851bc (127.1.212.60:41701): Couldn't send request to peer 0325606323134650a428f2ee67c851bc. Status: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 22:55:18.486485 9129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.487504 9228 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:55:18.487488 9129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.487577 9129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.487584 9228 log.cc:1196] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 22:55:18.487645 9129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.487709 9129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.487823 9228 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Deleting consensus metadata
I20260808 22:55:18.487977 9228 master_runner.cc:315] Copying system tablet from 127.1.212.62:34949
I20260808 22:55:18.488061 9182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.488684 9228 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: tablet copy: Beginning tablet copy session from remote peer at address 127.1.212.62:34949
I20260808 22:55:18.489353 9331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.489455 9331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:55:18.496170 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.489872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.1.212.62:34949)
0808 22:55:18.489924 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.489927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.490071 (+ 144us) server_negotiation.cc:407] Connection header received
0808 22:55:18.490154 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.490155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.490166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.490221 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.490578 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.491095 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.491920 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.492142 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.492225 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.493371 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.493372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.493373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.493387 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.494901 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.495295 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.495296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.495297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.495365 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.495593 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.495594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.495594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.495720 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.495799 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.495968 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.496060 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:55:18.496196 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.489803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:54316)
0808 22:55:18.489996 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.490012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.490100 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.490224 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.490225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.490439 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.490491 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.490493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.491195 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.491196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.491820 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.491822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.492226 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.492227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.492295 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.492873 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.492886 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.493273 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.495017 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.495018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.495019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.495219 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.495451 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.495452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.495452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.495523 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.495790 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.495791 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.495843 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.495939 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.496063 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260808 22:55:18.496824 8520 tablet_copy_service.cc:140] P f4620e1a99a84be798495865f881eea1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0325606323134650a428f2ee67c851bc ({username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:54316)
I20260808 22:55:18.496909 8520 tablet_copy_service.cc:161] P f4620e1a99a84be798495865f881eea1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0325606323134650a428f2ee67c851bc at {username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:54316: session id = 0325606323134650a428f2ee67c851bc-00000000000000000000000000000000
I20260808 22:55:18.497426 8520 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:55:18.497988 9228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:55:18.499048 9228 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: tablet copy: Starting download of 0 data blocks...
I20260808 22:55:18.499224 9228 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: tablet copy: Starting download of 1 WAL segments...
I20260808 22:55:18.500023 9228 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:55:18.500937 8520 tablet_copy_service.cc:342] P f4620e1a99a84be798495865f881eea1: Request end of tablet copy session 0325606323134650a428f2ee67c851bc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:54316
I20260808 22:55:18.501021 8520 tablet_copy_service.cc:433] P f4620e1a99a84be798495865f881eea1: ending tablet copy session 0325606323134650a428f2ee67c851bc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0325606323134650a428f2ee67c851bc
I20260808 22:55:18.503557 9228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.505004 9343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.505054 9342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.505430 9345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.509049 9228 server_base.cc:1034] running on GCE node
I20260808 22:55:18.509197 9228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.509433 9228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.510586 9228 hybrid_clock.cc:648] HybridClock initialized: now 1786229718510577 us; error 34 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.512158 9228 init.cc:377] Logged in from keytab as oryx/127.1.212.60@KRBTEST.COM (short username oryx)
I20260808 22:55:18.512507 9228 webserver.cc:460] Webserver started at http://127.1.212.60:45937/ using document root <none> and password file <none>
I20260808 22:55:18.512640 9228 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.512678 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.513412 9228 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.514006 9352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.514150 9228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:55:18.514204 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "0325606323134650a428f2ee67c851bc"
format_stamp: "Formatted at 2026-08-08 22:55:18 on dist-test-slave-1x4g"
server_key: "8e6a8a5ded95fbb24031bd9aab6a6438"
server_key_iv: "503dd6e763e7424a3a4e4cd9c5562e0c"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.514273 9228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.525600 9228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.527397 9228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.530814 9228 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.60:41701
I20260808 22:55:18.531287 9228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
I20260808 22:55:18.531633 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.60:41701 every 8 connection(s)
I20260808 22:55:18.532287 9409 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:55:18.532971 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Bootstrap starting.
I20260808 22:55:18.535588 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:55:18.535852 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Bootstrap complete.
I20260808 22:55:18.536084 9409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } }
I20260808 22:55:18.536182 9409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0325606323134650a428f2ee67c851bc, State: Initialized, Role: LEARNER
I20260808 22:55:18.536234 9409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } }
I20260808 22:55:18.536453 9409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:18.536418 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.536657 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: LEARNER
I20260808 22:55:18.538115 9427 catalog_manager.cc:1259] Loaded cluster ID: 62a5e3f51b3f40529bb6c28afe43a13c
I20260808 22:55:18.538677 9427 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: loading cluster ID for follower catalog manager: success
I20260808 22:55:18.539529 9427 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: acquiring CA information for follower catalog manager: success
I20260808 22:55:18.539862 9427 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 22:55:18.589257 9329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:18.589578 9329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:18.589629 9329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:18.591451 9329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:18.591557 9329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:18.591609 9329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:18.594091 9329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:35089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=35887
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:18.594413 9329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:18.594729 9329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.600857 9435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.600996 9434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.600867 9437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.601835 9329 server_base.cc:1034] running on GCE node
I20260808 22:55:18.602104 9329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.602419 9329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.603638 9329 hybrid_clock.cc:648] HybridClock initialized: now 1786229718603617 us; error 43 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.606822 9329 init.cc:377] Logged in from keytab as oryx/127.1.212.1@KRBTEST.COM (short username oryx)
I20260808 22:55:18.607486 9329 webserver.cc:460] Webserver started at http://127.1.212.1:35887/ using document root <none> and password file <none>
I20260808 22:55:18.607821 9329 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.607944 9329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.609967 9329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.611063 9447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.611254 9329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.611368 9329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "d7f6252b13bd4349b9cde0e74bcffcd8"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "9f3b76d6b9c2fcdcf58a603b94293976"
server_key_iv: "bb284480436030e2cf9743b7912cf11e"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.611759 9329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.623375 9329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.625916 9329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:18.626127 9329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.626413 9329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:18.626765 9329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:18.626819 9329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.626868 9329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:18.626891 9329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.634526 9329 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:35089
I20260808 22:55:18.634567 9576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:35089 every 8 connection(s)
I20260808 22:55:18.635004 9329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:18.635428 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 9329
I20260808 22:55:18.635639 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8715
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:18.645576 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:44937
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:34949
--webserver_port=46463
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.649679 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.636379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:57043 (local address 127.1.212.62:34949)
0808 22:55:18.636494 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.636497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.636700 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:55:18.637366 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.637368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.637382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.637444 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.638648 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.639322 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.641079 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.641362 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.641406 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.644547 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.644548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.644549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.644558 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.646607 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.647723 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.647725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.647726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.648068 (+ 342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.648488 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.648490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.648491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.648684 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.648802 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.649457 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.649577 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 22:55:18.650262 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.640134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:54207 (local address 127.1.212.60:41701)
0808 22:55:18.640551 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.640553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.640564 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:18.640591 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.640592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.640606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.640682 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.641211 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.642193 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.643953 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.644258 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.644330 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.646831 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.646833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.646834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.646844 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.648727 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.649263 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.649265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.649265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.649342 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.649575 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.649576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.649576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.649722 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.649787 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.650072 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.650166 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":69,"threads_started":1}
I20260808 22:55:18.650414 9582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.636404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.1.212.1:54207)
0808 22:55:18.640286 (+ 3882us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.640300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.640362 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.640738 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.640740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.641031 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.641125 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.641127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.642985 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.642996 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.643853 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.643857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.644402 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.644403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.644516 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.645030 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.645035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.646719 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.648891 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.648893 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.648893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.649145 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.649420 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.649421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.649422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.649498 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.649819 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.649820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.649878 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.649935 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.650131 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3856,"mutex_wait_us":20,"thread_start_us":82,"threads_started":1}
I20260808 22:55:18.651043 9578 heartbeater.cc:347] Connected to a master server at 127.1.212.60:41701
I20260808 22:55:18.651048 9581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.636052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:57043)
0808 22:55:18.636543 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.636607 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.637215 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.637551 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.637556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.638288 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.638528 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.638534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.639462 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.639464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.640956 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.640959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.641510 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.641511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.641665 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.642350 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.642359 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.644410 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.646720 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.646734 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.646738 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.647594 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.648171 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.648172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.648173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.648376 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.649040 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.649048 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.649251 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.649871 (+ 620us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.650922 (+ 1051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260808 22:55:18.651201 9578 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.651439 9578 heartbeater.cc:511] Master 127.1.212.60:41701 requested a full tablet report, sending...
I20260808 22:55:18.651482 9577 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:18.651542 9577 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.651594 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.640874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:48985 (local address 127.1.212.61:39075)
0808 22:55:18.640929 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.640932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.641186 (+ 254us) server_negotiation.cc:407] Connection header received
0808 22:55:18.641284 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.641285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.641296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.641325 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.641773 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.642939 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.644126 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.644457 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.644522 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.647881 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.647883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.647884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.647893 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.649970 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.650437 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.650439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.650439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.650522 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.650783 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.650784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.650784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.650963 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.651042 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.651377 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.651478 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:55:18.651636 9577 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:18.651758 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.636610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.1.212.1:48985)
0808 22:55:18.641122 (+ 4512us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.641133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.641211 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.641390 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.641391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.641597 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.641672 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.641675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.643084 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.643085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.643992 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.644008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.644614 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.644615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.645195 (+ 580us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.646010 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.646016 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.647752 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.650074 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.650075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.650076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.650309 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.650615 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.650616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.650616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.650691 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.651077 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.651078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.651128 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.651336 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.651463 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4492,"thread_start_us":67,"threads_started":1}
I20260808 22:55:18.651976 8493 ts_manager.cc:194] Re-registered known tserver with Master: d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1:35089)
I20260808 22:55:18.652424 9579 heartbeater.cc:347] Connected to a master server at 127.1.212.61:39075
I20260808 22:55:18.652503 9579 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.652462 9371 ts_manager.cc:194] Registered new tserver with Master: d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1:35089)
I20260808 22:55:18.652547 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:57043
I20260808 22:55:18.652576 9579 heartbeater.cc:511] Master 127.1.212.61:39075 requested a full tablet report, sending...
I20260808 22:55:18.653024 9171 ts_manager.cc:194] Registered new tserver with Master: d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1:35089)
W20260808 22:55:18.743229 9586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:18.743484 9586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:18.743512 9586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:18.744812 9586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:18.744858 9586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:18.744889 9586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:18.746584 9586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:44937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=46463
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:18.746855 9586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:18.747172 9586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.753688 9593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.753683 9594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.753683 9596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.753965 9586 server_base.cc:1034] running on GCE node
I20260808 22:55:18.754290 9586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.754588 9586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.755748 9586 hybrid_clock.cc:648] HybridClock initialized: now 1786229718755733 us; error 34 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.759089 9586 init.cc:377] Logged in from keytab as oryx/127.1.212.2@KRBTEST.COM (short username oryx)
I20260808 22:55:18.759639 9586 webserver.cc:460] Webserver started at http://127.1.212.2:46463/ using document root <none> and password file <none>
I20260808 22:55:18.759896 9586 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.759981 9586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.761729 9586 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:18.762591 9603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.762970 9586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.763106 9586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "d570333d395c4b39a63cbd700fdd6f50"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "1d6fe902e27c9b48c181f1f3298177ab"
server_key_iv: "26a938ffef92f6a67623f2c36cc14ed6"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.763507 9586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.781913 9586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.784549 9586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:18.784693 9586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.785045 9586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:18.785449 9586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:18.785488 9586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.785516 9586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:18.785528 9586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.793164 9586 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44937
I20260808 22:55:18.793221 9729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44937 every 8 connection(s)
I20260808 22:55:18.793642 9586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:18.798055 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 9586
I20260808 22:55:18.798250 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8865
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.806859 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.794682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:47623 (local address 127.1.212.60:41701)
0808 22:55:18.794769 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.794772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.795105 (+ 333us) server_negotiation.cc:407] Connection header received
0808 22:55:18.795712 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.795714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.795731 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.795786 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.796870 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797431 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.799061 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.799369 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.799409 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.802455 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.802457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.802457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.802468 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.804498 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.805073 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.805075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.805075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.805145 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.805424 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.805425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.805425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.805600 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.805669 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.806396 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.806749 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:55:18.808077 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.794597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.1.212.2:47623)
0808 22:55:18.794773 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.794819 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.795506 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.795812 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.795819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.796444 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.796634 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.796641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797570 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.797571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.798948 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.798952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.799503 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.799504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.799651 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.800353 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.800365 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.802322 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.804618 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.804632 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.804635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.804987 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.805222 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.805223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.805223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.805344 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.805700 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.805708 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.806204 (+ 496us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.807554 (+ 1350us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.807818 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":110,"thread_start_us":94,"threads_started":1}
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:18.809017 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.795033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:57479 (local address 127.1.212.61:39075)
0808 22:55:18.795079 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.795081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.795139 (+ 58us) server_negotiation.cc:407] Connection header received
0808 22:55:18.795662 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.795664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.795676 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.795709 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.796823 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797599 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.801195 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.801612 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.801667 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.805597 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.805599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.805600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.805610 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.807435 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.807999 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.808001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.808001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.808066 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.808283 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.808283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.808284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.808438 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.808484 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.808747 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.808873 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:55:18.809573 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.794952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.1.212.2:57479)
0808 22:55:18.795067 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.795089 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.795525 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.795828 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.795828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.796455 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.796634 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.796641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797725 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.797726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.801029 (+ 3303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.801035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.801768 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.801769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.802000 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.803634 (+ 1634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.803641 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.805487 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.807558 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.807559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.807560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.807906 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.808146 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.808147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.808147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.808212 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.808533 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.808534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.808630 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.809165 (+ 535us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.809453 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":46,"thread_start_us":74,"threads_started":1}
I20260808 22:55:18.810021 9732 heartbeater.cc:347] Connected to a master server at 127.1.212.61:39075
I20260808 22:55:18.810130 9732 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.810362 9732 heartbeater.cc:511] Master 127.1.212.61:39075 requested a full tablet report, sending...
I20260808 22:55:18.810448 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:36953
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:34949
--webserver_port=36169
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:55:18.810878 9731 heartbeater.cc:347] Connected to a master server at 127.1.212.60:41701
I20260808 22:55:18.810981 9731 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.811020 9171 ts_manager.cc:194] Registered new tserver with Master: d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:18.811139 9731 heartbeater.cc:511] Master 127.1.212.60:41701 requested a full tablet report, sending...
I20260808 22:55:18.811549 9371 ts_manager.cc:194] Registered new tserver with Master: d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:18.812708 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.794607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:60067 (local address 127.1.212.62:34949)
0808 22:55:18.795088 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.795090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.795100 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:18.795629 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.795630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.795645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.795712 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.796748 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797599 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.806507 (+ 8908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.806848 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.806922 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.809522 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.809523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.809523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.809534 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.811377 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.811815 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.811817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.811817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.811888 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.812144 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.812145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.812145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.812279 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.812362 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.812529 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.812614 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447}
I20260808 22:55:18.812724 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.794610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:60067)
0808 22:55:18.794862 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.794872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.795506 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.796137 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.796138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.796386 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.796633 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.796639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.797683 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.797684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.806411 (+ 8727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.806415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.807021 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.807022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.807108 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.807532 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.807538 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.809401 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.811474 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.811475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.811476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.811717 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.811963 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.811964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.811964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.812062 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.812352 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.812353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.812402 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.812493 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.812616 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":71,"thread_start_us":106,"threads_started":1}
I20260808 22:55:18.813064 9730 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:18.813122 9730 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.813174 9730 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:18.813395 8493 ts_manager.cc:194] Re-registered known tserver with Master: d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:18.813831 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:60067
W20260808 22:55:18.905015 9737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:18.905243 9737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:18.905270 9737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:18.906581 9737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:18.906637 9737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:18.906669 9737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:18.908731 9737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:36953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=36169
--tserver_master_addrs=127.1.212.62:34949,127.1.212.61:39075,127.1.212.60:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:18.909081 9737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:18.909372 9737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:18.915356 9746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.915432 9737 server_base.cc:1034] running on GCE node
W20260808 22:55:18.915343 9749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:18.915343 9747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:18.915848 9737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:18.916206 9737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:18.917395 9737 hybrid_clock.cc:648] HybridClock initialized: now 1786229718917364 us; error 45 us; skew 500 ppm
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:18.920296 9737 init.cc:377] Logged in from keytab as oryx/127.1.212.3@KRBTEST.COM (short username oryx)
I20260808 22:55:18.920894 9737 webserver.cc:460] Webserver started at http://127.1.212.3:36169/ using document root <none> and password file <none>
I20260808 22:55:18.921169 9737 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:18.921259 9737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:18.922794 9737 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:18.923534 9756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.923753 9737 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:18.923867 9737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "0c6b3632fbda4fc9bdacc093b913f782"
format_stamp: "Formatted at 2026-08-08 22:55:17 on dist-test-slave-1x4g"
server_key: "e549ff28c89222f9c69ce49b7e093bf6"
server_key_iv: "e5ee558fd459dee3c6423e54a9e4b1c9"
server_key_version: "encryptionkey@0"
I20260808 22:55:18.924270 9737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:18.946203 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.943424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.1.212.60:41701)
0808 22:55:18.943496 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.943499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.943513 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:55:18.943597 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.943598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.943633 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.943717 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:18.943879 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.944389 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.945436 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.945868 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.945961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.946057 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.946079 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.946117 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:18.946203 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.943323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:57004)
0808 22:55:18.943402 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.943419 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.943520 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.943726 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.943727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.943729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:18.943793 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.943796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.944500 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.944502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.945328 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.945330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.945963 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.945964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.946026 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.946030 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.946073 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.946122 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:55:18.946713 9381 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 22:55:18.946915 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4620e1a99a84be798495865f881eea1. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.947063 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: LEARNER
I20260808 22:55:18.949594 9737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:18.952179 9737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:18.952361 9737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:18.952641 9737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:18.953058 9737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:18.953112 9737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.953159 9737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:18.953197 9737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:18.960671 9737 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:36953
I20260808 22:55:18.960721 9882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:36953 every 8 connection(s)
I20260808 22:55:18.961184 9737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:18.967934 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 9737
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.60@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
Aug 08 22:55:18 dist-test-slave-1x4g krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229718, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.61@KRBTEST.COM
I20260808 22:55:18.979287 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:51573)
0808 22:55:18.962436 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.962487 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.963739 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.964081 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.964085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.964637 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.964938 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.964945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965951 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.965952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.966939 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.966941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967503 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.968136 (+ 632us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.968586 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.968591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.970613 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.974771 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.974772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.974772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.975098 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.976044 (+ 946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.976045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.976045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.976234 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.978105 (+ 1871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.978118 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.978580 (+ 462us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.978744 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.978998 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":55,"thread_start_us":91,"threads_started":1}
I20260808 22:55:18.980027 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:51573 (local address 127.1.212.62:34949)
0808 22:55:18.962371 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.962379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.962592 (+ 213us) server_negotiation.cc:407] Connection header received
0808 22:55:18.963890 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.963892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.963908 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.963944 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.965065 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965846 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967079 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967361 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967429 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.970706 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.970708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.970708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.970716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.974670 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.975187 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.975188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.975189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.975255 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.977817 (+ 2562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.977818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.977818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.977994 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.978062 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.979795 (+ 1733us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.979892 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:55:18.981318 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.970076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.1.212.61:39075)
0808 22:55:18.970898 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.970900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.970908 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:55:18.970934 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.970935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.970947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.971020 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.972859 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.975371 (+ 2512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.976358 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.976717 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.977085 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.978205 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.978206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.978207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.978214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.979881 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.980261 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.980262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.980262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.980321 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.980519 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.980519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.980520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.980653 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.980744 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.981131 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.981222 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":65,"threads_started":1}
I20260808 22:55:18.981961 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.1.212.3:53991)
0808 22:55:18.962752 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.962762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.963743 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.964263 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.964264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.964641 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.964938 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.964945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965951 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.965952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.966921 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.966925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967383 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.967462 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.969078 (+ 1616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.969083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.971703 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.980389 (+ 8686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.980390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.980391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.980641 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.980938 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.980938 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.980939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.981005 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.981329 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.981329 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.981628 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.981708 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.981834 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":33,"thread_start_us":74,"threads_started":1}
I20260808 22:55:18.982025 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:53991 (local address 127.1.212.61:39075)
0808 22:55:18.962757 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.962759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.962805 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:55:18.964162 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.964163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.964177 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.964250 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.965062 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965849 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967025 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967301 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967382 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.971818 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.971820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.971820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.971832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.973650 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.980755 (+ 7105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.980756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.980757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.980849 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.981084 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.981085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.981085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.981234 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.981290 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.981810 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.981920 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:55:18.982746 9883 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34949
I20260808 22:55:18.982863 9883 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.983120 9883 heartbeater.cc:511] Master 127.1.212.62:34949 requested a full tablet report, sending...
I20260808 22:55:18.983270 9885 heartbeater.cc:347] Connected to a master server at 127.1.212.61:39075
I20260808 22:55:18.983325 9885 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.983382 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.970446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.1.212.60:41701)
0808 22:55:18.970689 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.970692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.972659 (+ 1967us) server_negotiation.cc:407] Connection header received
0808 22:55:18.972691 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.972692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.972708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.972769 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.973909 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.974745 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.976434 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.976810 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.976878 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.978675 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.978678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.978678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.978687 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.981057 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.982033 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.982035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.982036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.982117 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.982325 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.982326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.982326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.982462 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.982566 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.983001 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.983276 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":65,"threads_started":1}
I20260808 22:55:18.983392 9885 heartbeater.cc:511] Master 127.1.212.61:39075 requested a full tablet report, sending...
I20260808 22:55:18.983546 8493 ts_manager.cc:194] Re-registered known tserver with Master: 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:18.983992 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.969214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.1.212.62:34949)
0808 22:55:18.969390 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.969392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.969696 (+ 304us) server_negotiation.cc:407] Connection header received
0808 22:55:18.969803 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.969803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.969818 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.969865 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.971205 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.972061 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.973280 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.973569 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.973613 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.975067 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.975068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.975068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.975076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.977713 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.982790 (+ 5077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.982791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.982791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.982866 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.983150 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.983151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.983152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.983335 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.983443 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.983806 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.983901 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":117,"threads_started":1}
I20260808 22:55:18.984069 8493 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:51573
I20260808 22:55:18.984117 9171 ts_manager.cc:194] Registered new tserver with Master: 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:18.985781 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.1.212.3:39161)
0808 22:55:18.962669 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.962678 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.963743 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.964111 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.964113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.964630 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:18.964937 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.964943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965694 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.965695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.966614 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.966618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967149 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:18.967303 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.968005 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:18.968018 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:18.970101 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:18.973029 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.973043 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.973046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.984749 (+ 11703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.985053 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:18.985054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:18.985054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:18.985127 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:18.985388 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:18.985389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:18.985436 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 22:55:18.985519 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:55:18.985652 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":153,"thread_start_us":86,"threads_started":1}
I20260808 22:55:18.985780 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.962111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:39161 (local address 127.1.212.60:41701)
0808 22:55:18.962170 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.962173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.962761 (+ 588us) server_negotiation.cc:407] Connection header received
0808 22:55:18.963891 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.963893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.963969 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.964087 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:18.965050 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.965573 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.966716 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.967059 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.967150 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:18.970237 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:18.970240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:18.970240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:18.970250 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:18.972921 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.984892 (+ 11971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.984893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.984894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.984961 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:18.985198 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:18.985199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:18.985200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:18.985318 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:18.985395 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:18.985564 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:55:18.985691 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:55:18.986119 9884 heartbeater.cc:347] Connected to a master server at 127.1.212.60:41701
I20260808 22:55:18.986174 9884 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:18.986263 9884 heartbeater.cc:511] Master 127.1.212.60:41701 requested a full tablet report, sending...
I20260808 22:55:18.986559 9371 ts_manager.cc:194] Registered new tserver with Master: 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:18.986235 8493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:18.988035 8493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:18.988924 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: NON_VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: true } } }
I20260808 22:55:18.989038 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.990487 9130 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: attempting to promote NON_VOTER 0325606323134650a428f2ee67c851bc to VOTER
I20260808 22:55:18.990619 9130 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } }
I20260808 22:55:18.991091 9381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 LEARNER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:55:18.991475 9181 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Refusing update from remote peer f4620e1a99a84be798495865f881eea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:55:18.991746 9130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 22:55:18.992136 9335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 22:55:18.992758 9130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0325606323134650a428f2ee67c851bc (127.1.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.992944 9381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0325606323134650a428f2ee67c851bc (127.1.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.994053 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.994045 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.994131 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:55:18.994136 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.994443 9182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0325606323134650a428f2ee67c851bc (127.1.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.995936 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.996037 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.996083 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.996117 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:18.997534 9909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4620e1a99a84be798495865f881eea1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:18.997663 9909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:55:19.001663 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.995704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35089 (local address 127.0.0.1:33072)
0808 22:55:18.997009 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.997016 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.997082 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.997297 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.997298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.997301 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:18.997363 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.997365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.998925 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.998926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.000111 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.000115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.001317 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.001318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.001396 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.001449 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 22:55:19.001509 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:55:19.001560 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"mutex_wait_us":41,"thread_start_us":765,"threads_started":1}
I20260808 22:55:19.002169 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.996103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.1.212.1:35089)
0808 22:55:18.996751 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.996757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.997084 (+ 327us) server_negotiation.cc:407] Connection header received
0808 22:55:18.997173 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.997176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.997197 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.997295 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:18.997449 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.998804 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.000237 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.001107 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.001190 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.001281 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.001811 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.001864 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":180,"threads_started":1}
I20260808 22:55:19.002691 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.995633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44937 (local address 127.0.0.1:47696)
0808 22:55:18.996854 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.996865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.996941 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.997463 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.997464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.997466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:18.997518 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.997519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.999225 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:18.999226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.000819 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.000823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.002398 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.002400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.002468 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.002473 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:19.002547 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:55:19.002604 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1190,"thread_start_us":113,"threads_started":1}
I20260808 22:55:19.002719 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.996006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.1.212.2:44937)
0808 22:55:18.997232 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.997237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.997247 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:18.997290 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.997292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.997308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.997386 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:18.997609 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:18.999075 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.001528 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.002285 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.002350 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.002407 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.002581 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.002627 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":118,"threads_started":1}
I20260808 22:55:19.003408 9655 tablet_service.cc:1467] Processing CreateTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f (DEFAULT_TABLE table=test-table [id=d451dc6bcf0149f9b57d4265c7745161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:19.003408 9503 tablet_service.cc:1467] Processing CreateTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f (DEFAULT_TABLE table=test-table [id=d451dc6bcf0149f9b57d4265c7745161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:19.003856 9655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421e09db594a4f16b9a2e754d26b0e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:19.003877 9503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421e09db594a4f16b9a2e754d26b0e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:19.005847 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.995598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36953 (local address 127.0.0.1:42762)
0808 22:55:18.995832 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:18.995858 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:18.995911 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:18.999514 (+ 3603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:18.999515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:18.999518 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:18.999564 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:18.999565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.001299 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.001300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.002363 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.002367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.005573 (+ 3206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.005574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.005644 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.005648 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:19.005719 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:55:19.005771 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260808 22:55:19.005865 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:18.996288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.1.212.3:36953)
0808 22:55:18.999300 (+ 3012us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:18.999305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:18.999318 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:18.999368 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:18.999370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:18.999390 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:18.999641 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:18.999713 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.001179 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.004921 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.005460 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.005572 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.005663 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.005736 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.005778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2961,"thread_start_us":97,"threads_started":1}
I20260808 22:55:19.008126 9916 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Bootstrap starting.
I20260808 22:55:19.008854 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f (DEFAULT_TABLE table=test-table [id=d451dc6bcf0149f9b57d4265c7745161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:19.009016 9916 tablet_bootstrap.cc:654] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:19.009276 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421e09db594a4f16b9a2e754d26b0e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:19.009356 9916 log.cc:824] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:19.009902 9915 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Bootstrap starting.
I20260808 22:55:19.010499 9916 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: No bootstrap required, opened a new log
I20260808 22:55:19.010596 9916 ts_tablet_manager.cc:1397] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:55:19.010612 9915 tablet_bootstrap.cc:654] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:19.010911 9915 log.cc:824] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:19.011922 9915 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: No bootstrap required, opened a new log
I20260808 22:55:19.012007 9915 ts_tablet_manager.cc:1397] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:19.013022 9916 raft_consensus.cc:348] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.013242 9916 raft_consensus.cc:374] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:19.013296 9916 raft_consensus.cc:729] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d570333d395c4b39a63cbd700fdd6f50, State: Initialized, Role: FOLLOWER
I20260808 22:55:19.013445 9916 consensus_queue.cc:260] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.013761 9919 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Bootstrap starting.
I20260808 22:55:19.013794 9916 ts_tablet_manager.cc:1428] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:55:19.013935 9730 heartbeater.cc:503] Master 127.1.212.62:34949 was elected leader, sending a full tablet report...
I20260808 22:55:19.014637 9919 tablet_bootstrap.cc:654] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:19.014222 9915 raft_consensus.cc:348] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.014957 9915 raft_consensus.cc:374] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:19.015014 9915 raft_consensus.cc:729] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f6252b13bd4349b9cde0e74bcffcd8, State: Initialized, Role: FOLLOWER
I20260808 22:55:19.015272 9919 log.cc:824] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:19.015513 9915 consensus_queue.cc:260] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.015910 9577 heartbeater.cc:503] Master 127.1.212.62:34949 was elected leader, sending a full tablet report...
I20260808 22:55:19.016139 9915 ts_tablet_manager.cc:1428] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 22:55:19.016295 9919 tablet_bootstrap.cc:492] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: No bootstrap required, opened a new log
I20260808 22:55:19.016358 9919 ts_tablet_manager.cc:1397] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:55:19.018240 9919 raft_consensus.cc:348] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.018396 9919 raft_consensus.cc:374] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:19.018420 9919 raft_consensus.cc:729] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b3632fbda4fc9bdacc093b913f782, State: Initialized, Role: FOLLOWER
I20260808 22:55:19.018564 9919 consensus_queue.cc:260] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.018898 9919 ts_tablet_manager.cc:1428] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:19.018956 9883 heartbeater.cc:503] Master 127.1.212.62:34949 was elected leader, sending a full tablet report...
W20260808 22:55:19.045077 9733 tablet.cc:2369] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:19.131804 9922 raft_consensus.cc:482] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:19.132145 9922 raft_consensus.cc:504] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.132613 9922 leader_election.cc:288] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953), d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:19.136888 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.133133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:47907 (local address 127.1.212.2:44937)
0808 22:55:19.133181 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.133183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.133193 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:19.133248 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.133249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.133259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.133329 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:19.133501 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.134269 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.135812 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.136384 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.136476 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.136489 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.136725 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.136811 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:55:19.136900 9581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.132995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44937 (local address 127.1.212.1:47907)
0808 22:55:19.133093 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:19.133103 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:19.133181 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:19.133335 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:19.133336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:19.133338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:19.133419 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.133422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.134375 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.134376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.135703 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.135705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.136504 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.136505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.136627 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.136632 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:19.136706 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:55:19.136765 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":9}
I20260808 22:55:19.136896 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.132987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:43271 (local address 127.1.212.3:36953)
0808 22:55:19.133037 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.133039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.133050 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:19.133083 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.133084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.133097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.133157 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:19.133710 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.134382 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.135811 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.136401 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.136476 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.136489 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.136761 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.136835 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:55:19.136914 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.132865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36953 (local address 127.1.212.1:43271)
0808 22:55:19.132948 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:19.132962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:19.133017 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:19.133185 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:19.133186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:19.133188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:19.133614 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.133617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.134476 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.134476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.135683 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:19.135688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.136504 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.136505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:19.136627 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.136632 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:19.136723 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:19.136765 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":29}
I20260808 22:55:19.137308 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421e09db594a4f16b9a2e754d26b0e1f" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d570333d395c4b39a63cbd700fdd6f50" is_pre_election: true
I20260808 22:55:19.137507 9676 raft_consensus.cc:2454] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 0.
I20260808 22:55:19.137477 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421e09db594a4f16b9a2e754d26b0e1f" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b3632fbda4fc9bdacc093b913f782" is_pre_election: true
I20260808 22:55:19.137655 9831 raft_consensus.cc:2454] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 0.
I20260808 22:55:19.137811 9449 leader_election.cc:302] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d570333d395c4b39a63cbd700fdd6f50, d7f6252b13bd4349b9cde0e74bcffcd8; no voters:
I20260808 22:55:19.137976 9922 raft_consensus.cc:2781] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:19.138075 9922 raft_consensus.cc:482] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:19.138098 9922 raft_consensus.cc:3037] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Advancing to term 1
W20260808 22:55:19.138526 9580 tablet.cc:2369] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:19.138981 9922 raft_consensus.cc:504] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.139183 9922 leader_election.cc:288] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 election: Requested vote from peers 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953), d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937)
I20260808 22:55:19.139366 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421e09db594a4f16b9a2e754d26b0e1f" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b3632fbda4fc9bdacc093b913f782"
I20260808 22:55:19.139405 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421e09db594a4f16b9a2e754d26b0e1f" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d570333d395c4b39a63cbd700fdd6f50"
I20260808 22:55:19.139467 9831 raft_consensus.cc:3037] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:19.139472 9676 raft_consensus.cc:3037] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:19.140295 9676 raft_consensus.cc:2454] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 1.
I20260808 22:55:19.140481 9449 leader_election.cc:302] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d570333d395c4b39a63cbd700fdd6f50, d7f6252b13bd4349b9cde0e74bcffcd8; no voters:
I20260808 22:55:19.140599 9922 raft_consensus.cc:2781] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:19.140748 9831 raft_consensus.cc:2454] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 1.
I20260808 22:55:19.140764 9922 raft_consensus.cc:686] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 LEADER]: Becoming Leader. State: Replica: d7f6252b13bd4349b9cde0e74bcffcd8, State: Running, Role: LEADER
I20260808 22:55:19.140992 9922 consensus_queue.cc:237] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:19.142277 8493 catalog_manager.cc:5495] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } health_report { overall_health: HEALTHY } } }
I20260808 22:55:19.149791 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.147124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.1.212.1:35089)
0808 22:55:19.147195 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.147197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.147207 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:19.147256 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.147257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.147277 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.147359 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:19.147509 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.148081 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.148906 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.149112 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.149186 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.149369 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:19.149488 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:19.149599 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.149687 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.149710 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:55:19.153766 9676 raft_consensus.cc:1261] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Refusing update from remote peer d7f6252b13bd4349b9cde0e74bcffcd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:19.153766 9831 raft_consensus.cc:1261] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Refusing update from remote peer d7f6252b13bd4349b9cde0e74bcffcd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:19.154214 9925 consensus_queue.cc:1035] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:19.154373 9922 consensus_queue.cc:1035] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:19.158017 9929 mvcc.cc:205] Tried to move back new op lower bound from 7316396929650442240 to 7316396929602248704. Current Snapshot: MvccSnapshot[applied={T|T < 7316396929650442240}]
I20260808 22:55:19.158209 9931 mvcc.cc:205] Tried to move back new op lower bound from 7316396929650442240 to 7316396929602248704. Current Snapshot: MvccSnapshot[applied={T|T < 7316396929650442240}]
I20260808 22:55:19.159598 9930 mvcc.cc:205] Tried to move back new op lower bound from 7316396929650442240 to 7316396929602248704. Current Snapshot: MvccSnapshot[applied={T|T < 7316396929650442240}]
I20260808 22:55:19.161787 8492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:19.161892 8492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:19.163525 8492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4620e1a99a84be798495865f881eea1: Sending DeleteTablet for 3 replicas of tablet 421e09db594a4f16b9a2e754d26b0e1f
I20260808 22:55:19.163944 9655 tablet_service.cc:1514] Processing DeleteTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:19 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:47696
I20260808 22:55:19.164023 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:19 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:42762
I20260808 22:55:19.164317 9939 tablet_replica.cc:330] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: stopping tablet replica
I20260808 22:55:19.164377 9503 tablet_service.cc:1514] Processing DeleteTablet for tablet 421e09db594a4f16b9a2e754d26b0e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:19 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:33072
I20260808 22:55:19.164439 9939 raft_consensus.cc:2229] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:19.164628 9939 raft_consensus.cc:2258] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:19.164930 9941 tablet_replica.cc:330] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: stopping tablet replica
I20260808 22:55:19.165038 9941 raft_consensus.cc:2229] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:19.165158 9941 raft_consensus.cc:2258] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:19.165370 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 8457
I20260808 22:55:19.166616 9939 ts_tablet_manager.cc:1905] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:19.168722 9939 ts_tablet_manager.cc:1918] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:19.168946 9939 log.cc:1196] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/421e09db594a4f16b9a2e754d26b0e1f
I20260808 22:55:19.169415 9939 ts_tablet_manager.cc:1939] T 421e09db594a4f16b9a2e754d26b0e1f P 0c6b3632fbda4fc9bdacc093b913f782: Deleting consensus metadata
I20260808 22:55:19.169644 9940 tablet_replica.cc:330] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: stopping tablet replica
I20260808 22:55:19.169799 9940 raft_consensus.cc:2229] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:19.169926 9940 raft_consensus.cc:2258] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:19.171363 9940 ts_tablet_manager.cc:1905] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:19.174407 9940 ts_tablet_manager.cc:1918] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:19.174638 9940 log.cc:1196] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/421e09db594a4f16b9a2e754d26b0e1f
I20260808 22:55:19.175029 9940 ts_tablet_manager.cc:1939] T 421e09db594a4f16b9a2e754d26b0e1f P d570333d395c4b39a63cbd700fdd6f50: Deleting consensus metadata
W20260808 22:55:19.176568 9155 connection.cc:535] server connection from 127.0.0.1:48186 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:55:19.176673 9761 connection.cc:535] server connection from 127.0.0.1:42762 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:55:19.177136 9606 connection.cc:535] server connection from 127.0.0.1:47696 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:55:19.178545 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.165446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.1.212.60:41701)
0808 22:55:19.165507 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.165510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.165984 (+ 474us) server_negotiation.cc:407] Connection header received
0808 22:55:19.166096 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.166097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.166109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.166140 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:19.170117 (+ 3977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.170917 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.172207 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.172538 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.172588 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.174430 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:19.174432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:19.174432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:19.174442 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:19.176284 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.177262 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.177264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.177264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.177331 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.177589 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.177591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.177591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.177784 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:19.177893 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.178360 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.178456 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:55:19.178624 9941 ts_tablet_manager.cc:1905] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:19.180611 9941 ts_tablet_manager.cc:1918] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:19.180702 9941 log.cc:1196] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/421e09db594a4f16b9a2e754d26b0e1f
I20260808 22:55:19.181031 9941 ts_tablet_manager.cc:1939] T 421e09db594a4f16b9a2e754d26b0e1f P d7f6252b13bd4349b9cde0e74bcffcd8: Deleting consensus metadata
W20260808 22:55:19.181443 9450 connection.cc:441] server connection from 127.0.0.1:33072 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33072 (request call id 1) could send its response
I20260808 22:55:19.187958 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.178566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.1.212.60:41701)
0808 22:55:19.178615 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.178618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.179193 (+ 575us) server_negotiation.cc:407] Connection header received
0808 22:55:19.179294 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.179294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.179310 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.179341 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:19.180579 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.181381 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.182400 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.182627 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.182713 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.184519 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:19.184521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:19.184521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:19.184531 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:19.186472 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.187035 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.187037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.187037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.187105 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.187337 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.187338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.187338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.187479 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:19.187590 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.187718 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.187816 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:55:19.188000 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:19.178220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.1.212.61:39075)
0808 22:55:19.178273 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:19.178275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:19.179159 (+ 884us) server_negotiation.cc:407] Connection header received
0808 22:55:19.179266 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:19.179267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:19.179280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:19.179311 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:19.180100 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.180946 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.182076 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:19.182402 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:19.182446 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:19.184621 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:19.184622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:19.184623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:19.184631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:19.186622 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.187077 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.187079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.187080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.187147 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:19.187357 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:19.187358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:19.187358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:19.187504 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:19.187591 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:19.187812 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 22:55:19.187902 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:55:20.159955 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.159568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:32853)
0808 22:55:20.159729 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.159825 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260808 22:55:20.160014 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.159568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:48317)
0808 22:55:20.159880 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.159965 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":58,"threads_started":1}
I20260808 22:55:20.160391 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.160075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:47373)
0808 22:55:20.160255 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.160339 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":57,"threads_started":1}
W20260808 22:55:20.160396 9883 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:34949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
W20260808 22:55:20.160562 9730 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:34949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
W20260808 22:55:20.160714 9577 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:34949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.664397 9955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4620e1a99a84be798495865f881eea1)
I20260808 22:55:20.664523 9955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } }
I20260808 22:55:20.664815 9955 leader_election.cc:288] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949), 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61:39075)
I20260808 22:55:20.664963 9956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4620e1a99a84be798495865f881eea1)
I20260808 22:55:20.665143 9956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } }
I20260808 22:55:20.665421 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:52972)
0808 22:55:20.665302 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.665367 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":56,"threads_started":1}
W20260808 22:55:20.665585 9354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.665892 9956 leader_election.cc:288] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949), 0325606323134650a428f2ee67c851bc (127.1.212.60:41701)
I20260808 22:55:20.666095 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:52980)
0808 22:55:20.666001 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.666056 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":75,"threads_started":1}
W20260808 22:55:20.666216 9155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.666301 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.666206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:52982)
0808 22:55:20.666245 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.666275 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 22:55:20.666468 9354 leader_election.cc:334] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949): Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.667501 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.666839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:52996)
0808 22:55:20.667394 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.667467 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537}
W20260808 22:55:20.667589 9155 leader_election.cc:334] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949): Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.669122 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:39075 (local address 127.0.0.1:60860)
0808 22:55:20.665341 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.665350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:20.665415 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:20.665633 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:20.665634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:20.665637 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:20.665698 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.665701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.666876 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.666882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.668072 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.668075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.668821 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.668823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.668889 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.668894 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:20.668971 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:55:20.669018 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":40,"threads_started":1}
I20260808 22:55:20.669132 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.1.212.61:39075)
0808 22:55:20.665258 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.665261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.665474 (+ 213us) server_negotiation.cc:407] Connection header received
0808 22:55:20.665506 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.665507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.665519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.665574 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:20.665780 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.666644 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.668152 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.668714 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.668822 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.668837 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.668978 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.669015 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":46,"threads_started":1}
I20260808 22:55:20.669385 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0325606323134650a428f2ee67c851bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5afb3209a4534ec88b758e874b74fbbd" is_pre_election: true
I20260808 22:55:20.669507 9182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0325606323134650a428f2ee67c851bc in term 1.
I20260808 22:55:20.669747 9356 leader_election.cc:302] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0325606323134650a428f2ee67c851bc, 5afb3209a4534ec88b758e874b74fbbd; no voters: f4620e1a99a84be798495865f881eea1
I20260808 22:55:20.669826 9965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.1.212.60:41701)
0808 22:55:20.666474 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.666476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.666486 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:20.666510 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.666510 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.666521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.666594 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:20.666753 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.667293 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.668707 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.669465 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.669538 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.669552 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.669716 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.669764 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":84,"threads_started":1}
I20260808 22:55:20.669876 9955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 22:55:20.669909 9955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Starting leader election (detected failure of leader f4620e1a99a84be798495865f881eea1)
I20260808 22:55:20.669927 9955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:55:20.670681 9955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } }
I20260808 22:55:20.670836 9955 leader_election.cc:288] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 election: Requested vote from peers f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949), 5afb3209a4534ec88b758e874b74fbbd (127.1.212.61:39075)
I20260808 22:55:20.669821 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.665683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41701 (local address 127.0.0.1:43412)
0808 22:55:20.666168 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.666177 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:20.666376 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:20.666622 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:20.666623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:20.666626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:20.666670 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.666671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.667392 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.667393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.668607 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.668609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.669549 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.669550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.669621 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.669626 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:20.669710 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:55:20.669754 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":43,"threads_started":1}
I20260808 22:55:20.671030 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.670932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:53004)
0808 22:55:20.670969 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.671012 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 22:55:20.671063 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0325606323134650a428f2ee67c851bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5afb3209a4534ec88b758e874b74fbbd"
I20260808 22:55:20.671144 9182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:55:20.671192 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5afb3209a4534ec88b758e874b74fbbd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0325606323134650a428f2ee67c851bc" is_pre_election: true
I20260808 22:55:20.671305 9381 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5afb3209a4534ec88b758e874b74fbbd in current term 2: Already voted for candidate 0325606323134650a428f2ee67c851bc in this term.
I20260808 22:55:20.671341 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.671248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:53016)
0808 22:55:20.671296 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.671321 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:55:20.671422 9354 leader_election.cc:334] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949): Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
I20260808 22:55:20.671552 9157 leader_election.cc:302] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5afb3209a4534ec88b758e874b74fbbd; no voters: 0325606323134650a428f2ee67c851bc, f4620e1a99a84be798495865f881eea1
I20260808 22:55:20.671993 9182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0325606323134650a428f2ee67c851bc in term 2.
I20260808 22:55:20.672114 9956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:55:20.672221 9356 leader_election.cc:302] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0325606323134650a428f2ee67c851bc, 5afb3209a4534ec88b758e874b74fbbd; no voters: f4620e1a99a84be798495865f881eea1
I20260808 22:55:20.672326 9955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:55:20.672482 9955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [term 2 LEADER]: Becoming Leader. State: Replica: 0325606323134650a428f2ee67c851bc, State: Running, Role: LEADER
I20260808 22:55:20.672576 9955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } }
I20260808 22:55:20.672967 9967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0325606323134650a428f2ee67c851bc. Latest consensus state: current_term: 2 leader_uuid: "0325606323134650a428f2ee67c851bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:20.673053 9967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:20.673233 9969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:20.673727 9969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:20.673791 9969 catalog_manager.cc:1259] Loaded cluster ID: 62a5e3f51b3f40529bb6c28afe43a13c
I20260808 22:55:20.673808 9969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:20.673943 9969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:20.673997 9969 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Loaded TSK: 0
I20260808 22:55:20.674088 9969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:20.727016 9370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:20.728893 9370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:20.730171 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.730030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:53026)
0808 22:55:20.730091 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.730136 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:55:20.730170 9182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [term 2 FOLLOWER]: Refusing update from remote peer 0325606323134650a428f2ee67c851bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 22:55:20.730490 9955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 22:55:20.730619 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.730501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.0.0.1:53028)
0808 22:55:20.730560 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.730596 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:55:20.730710 9354 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc -> Peer f4620e1a99a84be798495865f881eea1 (127.1.212.62:34949): Couldn't send request to peer f4620e1a99a84be798495865f881eea1. Status: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 22:55:20.731462 9956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0325606323134650a428f2ee67c851bc. Latest consensus state: current_term: 2 leader_uuid: "0325606323134650a428f2ee67c851bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:20.731570 9956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:55:20.732230 9967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0325606323134650a428f2ee67c851bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:20.732359 9967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:20.732316 9956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0325606323134650a428f2ee67c851bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f4620e1a99a84be798495865f881eea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34949 } } peers { permanent_uuid: "5afb3209a4534ec88b758e874b74fbbd" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 39075 } attrs { promote: false } } peers { permanent_uuid: "0325606323134650a428f2ee67c851bc" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41701 } attrs { promote: false } } }
I20260808 22:55:20.732409 9956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5afb3209a4534ec88b758e874b74fbbd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:55:20.738278 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48476 (local address 127.1.212.2:44937)
0808 22:55:20.734804 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.734806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.734813 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:55:20.734839 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.734840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.734853 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.735171 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:20.735216 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.736062 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.737346 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.737828 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.737977 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.737996 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.738138 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.738187 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260808 22:55:20.738277 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44937 (local address 127.0.0.1:48476)
0808 22:55:20.734578 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.734589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:20.734633 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:20.735029 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:20.735030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:20.735033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:20.735095 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.735097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.736165 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.736166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.737261 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.737264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.737976 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.737977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.738042 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.738047 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:20.738129 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:55:20.738187 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":38}
I20260808 22:55:20.738417 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.1.212.3:36953)
0808 22:55:20.735040 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.735043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.735051 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:55:20.735081 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.735082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.735094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.735306 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:20.735357 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.736069 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.737243 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.737855 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.738239 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.738256 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.738276 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.738325 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260808 22:55:20.738654 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36953 (local address 127.0.0.1:43570)
0808 22:55:20.734611 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.734624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:20.734674 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:20.735186 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:20.735187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:20.735189 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:20.735244 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.735245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.736149 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.736150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.737152 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.737154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.738024 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.738026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.738096 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.738100 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:20.738132 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 22:55:20.738187 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":44}
I20260808 22:55:20.738694 9655 tablet_service.cc:1467] Processing CreateTablet for tablet 593b760493874b83b81c748e4f5319ed (DEFAULT_TABLE table=test-table [id=ac82b1a9163b402c95b7e4cc204a7340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:20.738867 9655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593b760493874b83b81c748e4f5319ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:20.738926 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 593b760493874b83b81c748e4f5319ed (DEFAULT_TABLE table=test-table [id=ac82b1a9163b402c95b7e4cc204a7340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:20.739080 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593b760493874b83b81c748e4f5319ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:20.740159 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35089 (local address 127.0.0.1:45654)
0808 22:55:20.735012 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.735023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:20.735078 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:20.735473 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:20.735474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:20.735476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:20.735510 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.735511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.737286 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.737287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.739029 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:20.739033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.739848 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.739850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:20.739937 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.739944 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:55:20.740015 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:55:20.740075 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":94,"threads_started":1}
I20260808 22:55:20.740163 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.734810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.1.212.1:35089)
0808 22:55:20.735336 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.735338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.735346 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:55:20.735371 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.735372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.735381 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.735447 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:20.735581 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.736295 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.739149 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.739745 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.739837 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.739854 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.740038 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.740092 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":49,"threads_started":1}
I20260808 22:55:20.740677 9982 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Bootstrap starting.
I20260808 22:55:20.740697 9503 tablet_service.cc:1467] Processing CreateTablet for tablet 593b760493874b83b81c748e4f5319ed (DEFAULT_TABLE table=test-table [id=ac82b1a9163b402c95b7e4cc204a7340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:20.740846 9503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593b760493874b83b81c748e4f5319ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:20.740873 9983 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Bootstrap starting.
I20260808 22:55:20.741134 9982 tablet_bootstrap.cc:654] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:20.741252 9983 tablet_bootstrap.cc:654] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:20.741854 9982 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: No bootstrap required, opened a new log
I20260808 22:55:20.741894 9982 ts_tablet_manager.cc:1397] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:20.742046 9982 raft_consensus.cc:348] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.742129 9982 raft_consensus.cc:374] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:20.742152 9982 raft_consensus.cc:729] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d570333d395c4b39a63cbd700fdd6f50, State: Initialized, Role: FOLLOWER
I20260808 22:55:20.742198 9982 consensus_queue.cc:260] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.742285 9986 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: Bootstrap starting.
I20260808 22:55:20.742367 9982 ts_tablet_manager.cc:1428] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:55:20.742722 9983 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: No bootstrap required, opened a new log
I20260808 22:55:20.742790 9983 ts_tablet_manager.cc:1397] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:20.742904 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.742635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:56763)
0808 22:55:20.742813 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.742867 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260808 22:55:20.742980 9983 raft_consensus.cc:348] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.743060 9983 raft_consensus.cc:374] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:20.743080 9983 raft_consensus.cc:729] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6b3632fbda4fc9bdacc093b913f782, State: Initialized, Role: FOLLOWER
I20260808 22:55:20.743124 9983 consensus_queue.cc:260] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.743276 9983 ts_tablet_manager.cc:1428] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:55:20.743302 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.743124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:42937)
0808 22:55:20.743242 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.743270 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260808 22:55:20.743822 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.743479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:42401)
0808 22:55:20.743733 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.743773 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
I20260808 22:55:20.744143 9986 tablet_bootstrap.cc:654] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:20.744460 9884 heartbeater.cc:503] Master 127.1.212.60:41701 was elected leader, sending a full tablet report...
I20260808 22:55:20.744669 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.744572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:56623)
0808 22:55:20.744614 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.744638 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 22:55:20.745036 9986 tablet_bootstrap.cc:492] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: No bootstrap required, opened a new log
I20260808 22:55:20.745087 9986 ts_tablet_manager.cc:1397] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 22:55:20.745175 9986 raft_consensus.cc:348] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.745229 9986 raft_consensus.cc:374] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:20.745242 9986 raft_consensus.cc:729] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f6252b13bd4349b9cde0e74bcffcd8, State: Initialized, Role: FOLLOWER
I20260808 22:55:20.745307 9986 consensus_queue.cc:260] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.745457 9986 ts_tablet_manager.cc:1428] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:55:20.745903 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.745655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:49359)
0808 22:55:20.745797 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.745843 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260808 22:55:20.746109 9578 heartbeater.cc:503] Master 127.1.212.60:41701 was elected leader, sending a full tablet report...
I20260808 22:55:20.746331 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.746233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:45453)
0808 22:55:20.746272 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.746304 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 22:55:20.775820 9994 raft_consensus.cc:482] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:20.775931 9994 raft_consensus.cc:504] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.776176 9994 leader_election.cc:288] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937), 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:20.776477 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593b760493874b83b81c748e4f5319ed" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d570333d395c4b39a63cbd700fdd6f50" is_pre_election: true
I20260808 22:55:20.776476 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593b760493874b83b81c748e4f5319ed" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b3632fbda4fc9bdacc093b913f782" is_pre_election: true
I20260808 22:55:20.776611 9831 raft_consensus.cc:2454] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 0.
I20260808 22:55:20.776611 9676 raft_consensus.cc:2454] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 0.
I20260808 22:55:20.776844 9448 leader_election.cc:302] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b3632fbda4fc9bdacc093b913f782, d7f6252b13bd4349b9cde0e74bcffcd8; no voters:
I20260808 22:55:20.777029 9994 raft_consensus.cc:2781] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:20.777077 9994 raft_consensus.cc:482] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:20.777123 9994 raft_consensus.cc:3037] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:20.777951 9994 raft_consensus.cc:504] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.778116 9994 leader_election.cc:288] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 election: Requested vote from peers d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937), 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953)
I20260808 22:55:20.778290 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593b760493874b83b81c748e4f5319ed" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d570333d395c4b39a63cbd700fdd6f50"
I20260808 22:55:20.778306 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593b760493874b83b81c748e4f5319ed" candidate_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6b3632fbda4fc9bdacc093b913f782"
I20260808 22:55:20.778374 9831 raft_consensus.cc:3037] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:20.778374 9676 raft_consensus.cc:3037] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:20.779151 9676 raft_consensus.cc:2454] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 1.
I20260808 22:55:20.779152 9831 raft_consensus.cc:2454] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f6252b13bd4349b9cde0e74bcffcd8 in term 1.
I20260808 22:55:20.779366 9448 leader_election.cc:302] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6b3632fbda4fc9bdacc093b913f782, d7f6252b13bd4349b9cde0e74bcffcd8; no voters:
I20260808 22:55:20.779464 9994 raft_consensus.cc:2781] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:20.779629 9994 raft_consensus.cc:686] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 LEADER]: Becoming Leader. State: Replica: d7f6252b13bd4349b9cde0e74bcffcd8, State: Running, Role: LEADER
I20260808 22:55:20.779677 9994 consensus_queue.cc:237] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } }
I20260808 22:55:20.780144 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.779886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:48915)
0808 22:55:20.779928 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.780106 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 22:55:20.780398 9370 catalog_manager.cc:5495] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f6252b13bd4349b9cde0e74bcffcd8" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35089 } health_report { overall_health: HEALTHY } } }
I20260808 22:55:20.797802 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.795336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.1.212.1:35089)
0808 22:55:20.795401 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:20.795404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:20.795417 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:55:20.795447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:20.795448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:20.795462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:20.795521 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:20.795678 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.796181 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.797071 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:20.797285 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:20.797373 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:20.797515 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:20.797560 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:20.797622 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:20.797684 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 22:55:20.797713 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:55:20.798853 9831 raft_consensus.cc:1261] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Refusing update from remote peer d7f6252b13bd4349b9cde0e74bcffcd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:20.798867 9676 raft_consensus.cc:1261] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Refusing update from remote peer d7f6252b13bd4349b9cde0e74bcffcd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:20.799121 9996 consensus_queue.cc:1035] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6b3632fbda4fc9bdacc093b913f782" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:20.799255 9994 consensus_queue.cc:1035] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d570333d395c4b39a63cbd700fdd6f50" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:20.799819 9731 heartbeater.cc:503] Master 127.1.212.60:41701 was elected leader, sending a full tablet report...
I20260808 22:55:20.800238 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.800121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:38251)
0808 22:55:20.800168 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.800204 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:55:20.800395 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.800108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:55959)
0808 22:55:20.800315 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.800355 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260808 22:55:20.801724 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.801581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:56059)
0808 22:55:20.801651 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.801687 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:55:20.801774 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.801678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.3:38335)
0808 22:55:20.801718 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.801741 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 22:55:20.802029 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.801943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.1:44917)
0808 22:55:20.801977 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.801999 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 22:55:20.802085 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.801976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34949 (local address 127.1.212.2:35003)
0808 22:55:20.802019 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.802044 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 22:55:20.804234 9370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:20.804320 9370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:20.805639 9370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0325606323134650a428f2ee67c851bc: Sending DeleteTablet for 3 replicas of tablet 593b760493874b83b81c748e4f5319ed
I20260808 22:55:20.805984 9655 tablet_service.cc:1514] Processing DeleteTablet for tablet 593b760493874b83b81c748e4f5319ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:20 UTC) from {username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:48476
I20260808 22:55:20.806077 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet 593b760493874b83b81c748e4f5319ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:20 UTC) from {username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:43570
I20260808 22:55:20.806131 9503 tablet_service.cc:1514] Processing DeleteTablet for tablet 593b760493874b83b81c748e4f5319ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:20 UTC) from {username='oryx', principal='oryx/127.1.212.60@KRBTEST.COM'} at 127.0.0.1:45654
I20260808 22:55:20.806254 10014 tablet_replica.cc:330] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: stopping tablet replica
I20260808 22:55:20.806316 10014 raft_consensus.cc:2229] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:20.806362 10014 raft_consensus.cc:2258] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:20.806380 10016 tablet_replica.cc:330] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8: stopping tablet replica
I20260808 22:55:20.806438 10016 raft_consensus.cc:2229] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:20.806507 10016 raft_consensus.cc:2258] T 593b760493874b83b81c748e4f5319ed P d7f6252b13bd4349b9cde0e74bcffcd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:20.806620 10015 tablet_replica.cc:330] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: stopping tablet replica
I20260808 22:55:20.806680 10015 raft_consensus.cc:2229] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:20.806692 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 9329
I20260808 22:55:20.806730 10015 raft_consensus.cc:2258] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:20.807068 10015 ts_tablet_manager.cc:1905] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:20.810390 10015 ts_tablet_manager.cc:1918] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:20.810493 10015 log.cc:1196] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/593b760493874b83b81c748e4f5319ed
I20260808 22:55:20.810757 10015 ts_tablet_manager.cc:1939] T 593b760493874b83b81c748e4f5319ed P 0c6b3632fbda4fc9bdacc093b913f782: Deleting consensus metadata
I20260808 22:55:20.811225 9353 catalog_manager.cc:4841] TS 0c6b3632fbda4fc9bdacc093b913f782 (127.1.212.3:36953): tablet 593b760493874b83b81c748e4f5319ed (table test-table [id=ac82b1a9163b402c95b7e4cc204a7340]) successfully deleted
I20260808 22:55:20.813231 10014 ts_tablet_manager.cc:1905] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:20.815034 10014 ts_tablet_manager.cc:1918] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:20.815135 10014 log.cc:1196] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/593b760493874b83b81c748e4f5319ed
I20260808 22:55:20.815600 10014 ts_tablet_manager.cc:1939] T 593b760493874b83b81c748e4f5319ed P d570333d395c4b39a63cbd700fdd6f50: Deleting consensus metadata
I20260808 22:55:20.816058 9354 catalog_manager.cc:4841] TS d570333d395c4b39a63cbd700fdd6f50 (127.1.212.2:44937): tablet 593b760493874b83b81c748e4f5319ed (table test-table [id=ac82b1a9163b402c95b7e4cc204a7340]) successfully deleted
I20260808 22:55:20.817407 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:20.817252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35089 (local address 127.0.0.1:45686)
0808 22:55:20.817329 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:20.817365 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 22:55:20.817493 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 9586
W20260808 22:55:20.817644 9353 catalog_manager.cc:4584] TS d7f6252b13bd4349b9cde0e74bcffcd8 (127.1.212.1:35089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 593b760493874b83b81c748e4f5319ed: Network error: Client connection negotiation failed: client connection to 127.1.212.1:35089: connect: Connection refused (error 111)
I20260808 22:55:20.828421 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 9737
I20260808 22:55:20.839087 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 9018
I20260808 22:55:20.846745 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 9228
2026-08-08T22:55:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4914 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:20 dist-test-slave-1x4g krb5kdc[10020](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:20 dist-test-slave-1x4g krb5kdc[10020](info): set up 2 sockets
Aug 08 22:55:20 dist-test-slave-1x4g krb5kdc[10020](info): commencing operation
krb5kdc: starting...
W20260808 22:55:21.929320 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:22Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab.
Entry for principal oryx/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab.
I20260808 22:55:22.077091 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40221
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab
--principal=oryx/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:22.164345 10037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:22.164589 10037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:22.164615 10037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:22.166081 10037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:22.166141 10037 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:55:22.166157 10037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:22.166169 10037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:22.166182 10037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:22.166194 10037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:22.167969 10037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40221
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.62.keytab
--principal=oryx/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:22.168282 10037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:22.168551 10037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:22.174320 10043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.174358 10046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.174335 10044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:22.174371 10037 server_base.cc:1034] running on GCE node
I20260808 22:55:22.174935 10037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:22.175266 10037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:22.176460 10037 hybrid_clock.cc:648] HybridClock initialized: now 1786229722176438 us; error 47 us; skew 500 ppm
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:22.179533 10037 init.cc:377] Logged in from keytab as oryx/127.1.212.62@KRBTEST.COM (short username oryx)
I20260808 22:55:22.180122 10037 webserver.cc:460] Webserver started at http://127.1.212.62:40717/ using document root <none> and password file <none>
I20260808 22:55:22.180392 10037 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:22.180467 10037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:22.180629 10037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:22.181900 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "3ce25872566a4e31bd0660c01e41d0ad"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "b5d25e63f1075a19ab02b890b08d3b45"
server_key_iv: "ab532dc77ef09d33a27c426a124c0cc8"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.182322 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "3ce25872566a4e31bd0660c01e41d0ad"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "b5d25e63f1075a19ab02b890b08d3b45"
server_key_iv: "ab532dc77ef09d33a27c426a124c0cc8"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.183945 10037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:22.184906 10053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.185139 10037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:22.185278 10037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "3ce25872566a4e31bd0660c01e41d0ad"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "b5d25e63f1075a19ab02b890b08d3b45"
server_key_iv: "ab532dc77ef09d33a27c426a124c0cc8"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.185379 10037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:22.203027 10037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:22.205734 10037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:22.205951 10037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:22.211633 10037 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:40221
I20260808 22:55:22.211709 10123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:40221 every 8 connection(s)
I20260808 22:55:22.212146 10037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:22.213059 10124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:22.213835 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10037
I20260808 22:55:22.213971 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:22.214219 1872 external_mini_cluster.cc:1428] Setting key 9ff87449db2d7033812892ba9aa7116f
I20260808 22:55:22.216634 10124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Bootstrap starting.
I20260808 22:55:22.217535 10124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:22.218225 10124 log.cc:824] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:22.219152 10124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: No bootstrap required, opened a new log
I20260808 22:55:22.221896 10124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } }
I20260808 22:55:22.222114 10124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:22.222159 10124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce25872566a4e31bd0660c01e41d0ad, State: Initialized, Role: FOLLOWER
I20260808 22:55:22.222313 10124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } }
I20260808 22:55:22.222405 10124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:22.222446 10124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:22.222502 10124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 0 FOLLOWER]: Advancing to term 1
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:22.223426 10124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } }
I20260808 22:55:22.223534 10124 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce25872566a4e31bd0660c01e41d0ad; no voters:
I20260808 22:55:22.223768 10124 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:22.223824 10130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:22.224120 10124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:22.224449 10130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [term 1 LEADER]: Becoming Leader. State: Replica: 3ce25872566a4e31bd0660c01e41d0ad, State: Running, Role: LEADER
I20260808 22:55:22.224587 10130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } }
I20260808 22:55:22.225402 10132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce25872566a4e31bd0660c01e41d0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } } }
I20260808 22:55:22.225507 10132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:22.225634 10131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce25872566a4e31bd0660c01e41d0ad. Latest consensus state: current_term: 1 leader_uuid: "3ce25872566a4e31bd0660c01e41d0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce25872566a4e31bd0660c01e41d0ad" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40221 } } }
I20260808 22:55:22.225967 10131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad [sys.catalog]: This master's current role is: LEADER
W20260808 22:55:22.226361 10144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:22.226428 10144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:22.226511 10145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:22.226724 10145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:22.228123 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.214933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.1.212.62:40221)
0808 22:55:22.215157 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.215168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.215202 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:55:22.215741 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.215752 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.215826 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.215996 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.216730 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.218498 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.219616 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.220257 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.220413 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.223345 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.223350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.223350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.223365 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.225899 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.226600 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.226604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.226604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.226746 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.227007 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.227009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.227009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.227379 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.227567 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.227732 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.227900 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":106,"threads_started":1}
I20260808 22:55:22.229007 10145 catalog_manager.cc:1347] Generated new cluster ID: 38b46c8ca3cf4925b94ecca2b7490f68
I20260808 22:55:22.229071 10145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:22.243191 10145 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:22.243826 10145 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:22.252316 10145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Generated new TSK 0
I20260808 22:55:22.252564 10145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
I20260808 22:55:22.292517 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40221
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:22.394713 10152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:22.394980 10152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:22.395011 10152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:22.396381 10152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:22.396450 10152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:22.396474 10152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:55:22.396494 10152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:22.398347 10152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:40221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:22.398661 10152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:22.398947 10152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:22.404565 10159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.404562 10161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.404562 10158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:22.405133 10152 server_base.cc:1034] running on GCE node
I20260808 22:55:22.405400 10152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:22.405684 10152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:22.406868 10152 hybrid_clock.cc:648] HybridClock initialized: now 1786229722406858 us; error 38 us; skew 500 ppm
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:22.409875 10152 init.cc:377] Logged in from keytab as oryx/127.1.212.1@KRBTEST.COM (short username oryx)
I20260808 22:55:22.410493 10152 webserver.cc:460] Webserver started at http://127.1.212.1:41693/ using document root <none> and password file <none>
I20260808 22:55:22.410746 10152 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:22.410820 10152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:22.410988 10152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:22.412196 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "4d8994928dd545528e2f6f2441c8c4c0"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "1feff2590decae0cb5ec1cd971a18b30"
server_key_iv: "47eba999397b77357fa30d04b9503c9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.412622 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "4d8994928dd545528e2f6f2441c8c4c0"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "1feff2590decae0cb5ec1cd971a18b30"
server_key_iv: "47eba999397b77357fa30d04b9503c9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.414309 10152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:22.415272 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.415537 10152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:22.415661 10152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "4d8994928dd545528e2f6f2441c8c4c0"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "1feff2590decae0cb5ec1cd971a18b30"
server_key_iv: "47eba999397b77357fa30d04b9503c9f"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.415755 10152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:22.427186 10152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:22.430552 10152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:22.430781 10152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:22.431603 10152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:22.431667 10152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.431705 10152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:22.431730 10152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:22.440387 10152 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:38085
I20260808 22:55:22.440601 10293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:38085 every 8 connection(s)
I20260808 22:55:22.440968 10152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:22.443338 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.432184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:48325 (local address 127.1.212.62:40221)
0808 22:55:22.432269 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.432272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.432518 (+ 246us) server_negotiation.cc:407] Connection header received
0808 22:55:22.433184 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.433185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.433200 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.433262 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.434084 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.434668 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.436587 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.437046 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.437160 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.439820 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.439822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.439823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.439835 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.441813 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.442271 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.442272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.442272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.442337 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.442601 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.442602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.442602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.442724 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.442779 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.443155 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.443266 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:55:22.443372 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.432175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.1:48325)
0808 22:55:22.432362 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.432406 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.433059 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.433347 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.433351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.433716 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.433939 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.433945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.434786 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.434787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.436426 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.436431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.437203 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.437204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.437360 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.438026 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.438049 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.439694 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.441910 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.441915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.441918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.442185 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.442408 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.442408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.442409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.442540 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.442785 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.442790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.442931 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.443028 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.443181 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260808 22:55:22.444264 10294 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40221
I20260808 22:55:22.444394 10294 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:22.444604 10294 heartbeater.cc:511] Master 127.1.212.62:40221 requested a full tablet report, sending...
I20260808 22:55:22.445328 10072 ts_manager.cc:194] Registered new tserver with Master: 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085)
I20260808 22:55:22.446106 10072 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:48325
I20260808 22:55:22.451171 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10152
I20260808 22:55:22.451321 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:22.451508 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.445608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.1:51845)
0808 22:55:22.445794 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.445804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.445888 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.446028 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.446028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.446268 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.446387 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.446390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.447063 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.447064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.447606 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.447608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.448004 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.448005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.448080 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.448552 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.448557 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.448972 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.450365 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.450367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.450367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.450556 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.450778 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.450779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.450780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.450863 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.451139 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.451141 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.451189 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.451302 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.451432 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260808 22:55:22.451560 1872 external_mini_cluster.cc:1428] Setting key 35c5d87327c684269fc636f35b8ba11a
I20260808 22:55:22.451684 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.445683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51845 (local address 127.1.212.62:40221)
0808 22:55:22.445732 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.445734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.445868 (+ 134us) server_negotiation.cc:407] Connection header received
0808 22:55:22.445960 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.445961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.445972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.446024 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.446466 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.446963 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.447701 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.447907 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.447999 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.449068 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.449069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.449070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.449078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.450273 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.450648 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.450649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.450650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.450711 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.450944 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.450945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.450945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.451066 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.451137 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.451492 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.451593 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
I20260808 22:55:22.483283 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40221
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:22.571973 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:22.572247 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:22.572293 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:22.573617 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:22.573702 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:22.573740 10303 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:55:22.573782 10303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:22.575515 10303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:40221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:22.575846 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:22.576118 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:22.581831 10312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.581825 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.581825 10310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:22.582111 10303 server_base.cc:1034] running on GCE node
I20260808 22:55:22.582357 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:22.582650 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:22.583834 10303 hybrid_clock.cc:648] HybridClock initialized: now 1786229722583817 us; error 48 us; skew 500 ppm
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:22.586764 10303 init.cc:377] Logged in from keytab as oryx/127.1.212.2@KRBTEST.COM (short username oryx)
I20260808 22:55:22.587317 10303 webserver.cc:460] Webserver started at http://127.1.212.2:41463/ using document root <none> and password file <none>
I20260808 22:55:22.587598 10303 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:22.587672 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:22.587833 10303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:22.589345 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "34b95da504f31c47a3619422a8c121ec"
server_key_iv: "059db62f0dbcde4ff9791b5ae14b3de5"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.589759 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "34b95da504f31c47a3619422a8c121ec"
server_key_iv: "059db62f0dbcde4ff9791b5ae14b3de5"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.591275 10303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:22.592114 10319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.592324 10303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:22.592437 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "34b95da504f31c47a3619422a8c121ec"
server_key_iv: "059db62f0dbcde4ff9791b5ae14b3de5"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.592518 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:22.602330 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:22.604660 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:22.604883 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:22.605573 10303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:22.605638 10303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.605675 10303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:22.605700 10303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:22.614010 10303 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:46149
I20260808 22:55:22.614158 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:46149 every 8 connection(s)
I20260808 22:55:22.614502 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:22.616844 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.606041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:34527 (local address 127.1.212.62:40221)
0808 22:55:22.606119 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.606121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.606386 (+ 265us) server_negotiation.cc:407] Connection header received
0808 22:55:22.606917 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.606919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.606935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.607027 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.607825 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.608706 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.610097 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.610338 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.610429 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.613610 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.613612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.613612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.613621 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.615405 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.615857 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.615858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.615858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.615918 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.616154 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.616154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.616154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.616267 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.616316 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.616625 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.616702 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:55:22.617060 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.606046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.2:34527)
0808 22:55:22.606227 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.606270 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.606798 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.607145 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.607149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.607494 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.607710 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.607716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.608874 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.608874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.609985 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.609990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.610469 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.610471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.610620 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.611321 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.611335 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.613467 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.615511 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.615517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.615519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.615774 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.615996 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.615997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.615997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.616109 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.616320 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.616325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.616487 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.616580 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.616739 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260808 22:55:22.617652 10449 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40221
I20260808 22:55:22.617750 10449 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:22.617983 10449 heartbeater.cc:511] Master 127.1.212.62:40221 requested a full tablet report, sending...
I20260808 22:55:22.618464 10071 ts_manager.cc:194] Registered new tserver with Master: 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:22.618868 10071 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:34527
I20260808 22:55:22.619998 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10303
I20260808 22:55:22.620122 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:22.620322 1872 external_mini_cluster.cc:1428] Setting key 1e93778f2ed9366d894bbe0882eb0bc6
I20260808 22:55:22.626282 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.618901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.2:46007)
0808 22:55:22.619129 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.619147 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.619229 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.619363 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.619363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.619624 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.619700 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.619702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.620692 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.620693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.621546 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.621549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.621920 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.621921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.622017 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.622509 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.622515 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.623006 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.625049 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.625050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.625051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.625245 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.625469 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.625470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.625470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.625518 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.625977 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.625977 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.626029 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.626086 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.626192 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260808 22:55:22.626398 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.618999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46007 (local address 127.1.212.62:40221)
0808 22:55:22.619056 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.619059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.619210 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:55:22.619292 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.619293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.619305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.619358 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.619792 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.620558 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.621642 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.621833 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.621892 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.623113 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.623115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.623115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.623126 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.624942 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.625324 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.625325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.625326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.625400 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.625648 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.625649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.625649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.625888 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.625964 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.626143 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.626254 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
WARNING: no policy specified for oryx/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
I20260808 22:55:22.652735 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:40221
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:22.741330 10458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:22.741621 10458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:22.741668 10458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:22.743053 10458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:22.743135 10458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:22.743177 10458 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:55:22.743238 10458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:22.745222 10458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.212.62:40221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:22.745553 10458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:22.745823 10458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:22.751442 10466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.751436 10469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:22.751442 10467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:22.751713 10458 server_base.cc:1034] running on GCE node
I20260808 22:55:22.752007 10458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:22.752264 10458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:22.753453 10458 hybrid_clock.cc:648] HybridClock initialized: now 1786229722753431 us; error 44 us; skew 500 ppm
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:22.756443 10458 init.cc:377] Logged in from keytab as oryx/127.1.212.3@KRBTEST.COM (short username oryx)
I20260808 22:55:22.757052 10458 webserver.cc:460] Webserver started at http://127.1.212.3:39461/ using document root <none> and password file <none>
I20260808 22:55:22.757304 10458 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:22.757378 10458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:22.757539 10458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:22.758793 10458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "ede8c626dc604d2d9acb3bee5c168deb"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "21b3d3d48f80a9617ff5773b0a07174b"
server_key_iv: "b2b25389f63caf251b6d3eabf4447e02"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.759212 10458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "ede8c626dc604d2d9acb3bee5c168deb"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "21b3d3d48f80a9617ff5773b0a07174b"
server_key_iv: "b2b25389f63caf251b6d3eabf4447e02"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.760862 10458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 22:55:22.761693 10476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.761895 10458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:22.762013 10458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "ede8c626dc604d2d9acb3bee5c168deb"
format_stamp: "Formatted at 2026-08-08 22:55:22 on dist-test-slave-1x4g"
server_key: "21b3d3d48f80a9617ff5773b0a07174b"
server_key_iv: "b2b25389f63caf251b6d3eabf4447e02"
server_key_version: "encryptionkey@0"
I20260808 22:55:22.762097 10458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:22.774207 10458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:22.776265 10458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:22.776448 10458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:22.777066 10458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:22.777138 10458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.777189 10458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:22.777215 10458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:22.784663 10458 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:36643
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:22.784751 10601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:36643 every 8 connection(s)
I20260808 22:55:22.785110 10458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:22.788146 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.777619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:35179 (local address 127.1.212.62:40221)
0808 22:55:22.777663 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.777666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.777997 (+ 331us) server_negotiation.cc:407] Connection header received
0808 22:55:22.778633 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.778635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.778651 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.778729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.779740 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.780615 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.782367 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.782615 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.782696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.785357 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.785358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.785359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.785367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.786638 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.787091 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.787093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.787093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.787164 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.787442 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.787443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.787443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.787577 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.787653 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.787949 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.788042 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:55:22.788357 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.777610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.3:35179)
0808 22:55:22.777831 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.777884 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.778441 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.778837 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.778842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.779292 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.779610 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.779617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.780752 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.780753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.782266 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.782273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.782724 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.782725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.782869 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.783581 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.783593 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.785237 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.786743 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.786751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.786754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.786997 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.787236 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.787237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.787238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.787363 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.787655 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.787664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.787814 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.787927 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.788105 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260808 22:55:22.788888 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10458
I20260808 22:55:22.788995 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:22.788970 10602 heartbeater.cc:347] Connected to a master server at 127.1.212.62:40221
I20260808 22:55:22.789060 10602 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:22.789212 1872 external_mini_cluster.cc:1428] Setting key 0b99f9fea5aa834b55df5d11202d3d61
I20260808 22:55:22.789278 10602 heartbeater.cc:511] Master 127.1.212.62:40221 requested a full tablet report, sending...
I20260808 22:55:22.790195 10072 ts_manager.cc:194] Registered new tserver with Master: ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:22.790599 10072 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:35179
I20260808 22:55:22.795821 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.790294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:50157 (local address 127.1.212.62:40221)
0808 22:55:22.790334 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.790336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.790494 (+ 158us) server_negotiation.cc:407] Connection header received
0808 22:55:22.790559 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.790559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.790571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.790626 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.790995 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.791490 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.792243 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.792454 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.792531 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.793481 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.793482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.793483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.793490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.794605 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.794961 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.794962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.794963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.795024 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.795244 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.795245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.795245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.795369 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.795443 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.795629 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.795718 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:55:22.795840 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.790245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.1.212.3:50157)
0808 22:55:22.790416 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.790426 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.790510 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.790631 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.790632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.790843 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:22.790921 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.790924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.791602 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.791603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.792139 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.792141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.792532 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.792533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.792614 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.793088 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:22.793093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:22.793390 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:22.794694 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.794695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.794695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.794875 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.795114 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:22.795115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:22.795115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:22.795181 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:22.795447 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:22.795448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:22.795498 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.795596 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.795734 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":120,"threads_started":1}
I20260808 22:55:22.801195 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:55:22.825241 10128 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:22.819305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.1.212.62:40221)
0808 22:55:22.819401 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.819404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.819416 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:55:22.819445 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.819446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.819459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.819526 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.820168 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.820737 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.821550 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.821741 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.821797 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.825132 (+ 3335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53052: BlockingRecv error: recv got EOF from 127.0.0.1:53052 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 22:55:22 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:22.835355 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.826015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.1.212.62:40221)
0808 22:55:22.826083 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.826085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.826154 (+ 69us) server_negotiation.cc:407] Connection header received
0808 22:55:22.826228 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.826228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.826239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.826296 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:22.826815 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.827731 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.828454 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.828670 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.828753 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.832156 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:22.832158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:22.832158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:22.832168 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:22.833998 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.834482 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.834484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.834485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.834578 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:22.834832 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:22.834833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:22.834834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:22.835018 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:22.835106 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.835212 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.835284 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:55:22.837476 10072 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:22.839186 10072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:22.846179 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.841823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.0.0.1:43140)
0808 22:55:22.842001 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.842045 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.842123 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.842403 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.842406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.842409 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:22.842498 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.842500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.843629 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.843630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.844865 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.844868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.845825 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.845826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.845928 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.845945 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.846015 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.846079 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":58,"threads_started":1}
I20260808 22:55:22.846185 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.841996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.1.212.3:36643)
0808 22:55:22.842209 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.842214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.842225 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:22.842272 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.842275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.842291 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.842400 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:22.842682 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.843533 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.844954 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.845723 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.845812 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.845908 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.846041 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.846082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260808 22:55:22.846555 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.842043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:46149 (local address 127.0.0.1:60310)
0808 22:55:22.842286 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.842296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.842346 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.842668 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.842669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.842672 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:22.842731 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.842734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.844231 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.844233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.845407 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.845410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.846306 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.846308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.846391 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.846397 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.846426 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.846481 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260808 22:55:22.846849 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.841828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38085 (local address 127.0.0.1:43086)
0808 22:55:22.842176 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:22.842188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:22.842249 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:22.842860 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:22.842861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:22.842863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:22.842913 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.842914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.844202 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.844203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.845322 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:22.845325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.846620 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.846621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:22.846689 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.846693 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:22.846737 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 22:55:22.846776 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":106,"threads_started":1}
I20260808 22:55:22.847111 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.842285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.1.212.2:46149)
0808 22:55:22.842486 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.842490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.842500 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:22.842542 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.842543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.842562 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.842839 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:22.842911 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.844129 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.845496 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.846205 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.846750 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.846851 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.846880 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.847003 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260808 22:55:22.847062 10530 tablet_service.cc:1467] Processing CreateTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 (DEFAULT_TABLE table=test-table [id=6bdea425a18c4841bf2bdfff52020d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:22.847256 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:22.841990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.1.212.1:38085)
0808 22:55:22.842542 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:22.842547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:22.842556 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:22.842600 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:22.842602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:22.842618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:22.842806 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:22.843012 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.844106 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.845407 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:22.846164 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:22.846973 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:22.847072 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:22.847098 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 22:55:22.847150 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":99,"threads_started":1}
I20260808 22:55:22.847517 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4a4dfcc215e439a81a67e85c47c4c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:22.847877 10224 tablet_service.cc:1467] Processing CreateTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 (DEFAULT_TABLE table=test-table [id=6bdea425a18c4841bf2bdfff52020d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:22.848212 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4a4dfcc215e439a81a67e85c47c4c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:22.849279 10376 tablet_service.cc:1467] Processing CreateTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 (DEFAULT_TABLE table=test-table [id=6bdea425a18c4841bf2bdfff52020d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:22.849638 10376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4a4dfcc215e439a81a67e85c47c4c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:22.851290 10632 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Bootstrap starting.
I20260808 22:55:22.851714 10633 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Bootstrap starting.
I20260808 22:55:22.852144 10632 tablet_bootstrap.cc:654] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:22.852424 10633 tablet_bootstrap.cc:654] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:22.852497 10632 log.cc:824] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:22.852878 10633 log.cc:824] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:22.853438 10632 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: No bootstrap required, opened a new log
I20260808 22:55:22.853535 10632 ts_tablet_manager.cc:1397] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:22.853749 10633 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: No bootstrap required, opened a new log
I20260808 22:55:22.853825 10633 ts_tablet_manager.cc:1397] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:55:22.854090 10637 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Bootstrap starting.
I20260808 22:55:22.854821 10637 tablet_bootstrap.cc:654] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:22.855185 10637 log.cc:824] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:22.855859 10633 raft_consensus.cc:348] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.855842 10632 raft_consensus.cc:348] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.856037 10633 raft_consensus.cc:374] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:22.856037 10632 raft_consensus.cc:374] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:22.856081 10637 tablet_bootstrap.cc:492] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: No bootstrap required, opened a new log
I20260808 22:55:22.856084 10633 raft_consensus.cc:729] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Initialized, Role: FOLLOWER
I20260808 22:55:22.856083 10632 raft_consensus.cc:729] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Initialized, Role: FOLLOWER
I20260808 22:55:22.856141 10637 ts_tablet_manager.cc:1397] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:22.856222 10632 consensus_queue.cc:260] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.856259 10633 consensus_queue.cc:260] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.856545 10632 ts_tablet_manager.cc:1428] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:55:22.856590 10294 heartbeater.cc:503] Master 127.1.212.62:40221 was elected leader, sending a full tablet report...
I20260808 22:55:22.856688 10602 heartbeater.cc:503] Master 127.1.212.62:40221 was elected leader, sending a full tablet report...
I20260808 22:55:22.856575 10633 ts_tablet_manager.cc:1428] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:55:22.858446 10637 raft_consensus.cc:348] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.858588 10637 raft_consensus.cc:374] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:22.858610 10637 raft_consensus.cc:729] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Initialized, Role: FOLLOWER
I20260808 22:55:22.858726 10637 consensus_queue.cc:260] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:22.858973 10637 ts_tablet_manager.cc:1428] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:22.859037 10449 heartbeater.cc:503] Master 127.1.212.62:40221 was elected leader, sending a full tablet report...
W20260808 22:55:22.865875 10450 tablet.cc:2369] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:55:22.941814 10295 tablet.cc:2369] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:23.000989 10640 raft_consensus.cc:482] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:23.001184 10640 raft_consensus.cc:504] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:23.001603 10640 leader_election.cc:288] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.005861 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.001992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:44955 (local address 127.1.212.2:46149)
0808 22:55:23.002053 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.002055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.002099 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:55:23.002127 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.002128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.002141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.002193 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.002438 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.003194 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.004876 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.005440 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.005530 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.005540 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.005738 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.005783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:55:23.005861 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.001964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:55887 (local address 127.1.212.3:36643)
0808 22:55:23.002042 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.002044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.002058 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:55:23.002096 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.002098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.002111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.002179 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.002439 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.003210 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.004894 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.005444 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.005525 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.005538 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.005741 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.005782 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:55:23.005914 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.001818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.1.212.1:55887)
0808 22:55:23.001957 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.001970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.002030 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.002216 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.002217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.002222 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.002345 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.002346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.003295 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.003296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.004744 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.004750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.005542 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.005544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.005647 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.005653 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.005717 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.005783 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":96,"thread_start_us":63,"threads_started":1}
I20260808 22:55:23.005919 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.001818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:46149 (local address 127.1.212.1:44955)
0808 22:55:23.001958 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.001970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.002030 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.002216 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.002217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.002220 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.002343 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.002346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.003294 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.003295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.004745 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.004750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.005545 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.005546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.005659 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.005662 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.005717 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.005783 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":158,"thread_start_us":72,"threads_started":1}
I20260808 22:55:23.006404 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4a4dfcc215e439a81a67e85c47c4c12" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" is_pre_election: true
I20260808 22:55:23.006516 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4a4dfcc215e439a81a67e85c47c4c12" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb" is_pre_election: true
I20260808 22:55:23.006610 10398 raft_consensus.cc:2454] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 0.
I20260808 22:55:23.006657 10552 raft_consensus.cc:2454] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 0.
I20260808 22:55:23.006878 10174 leader_election.cc:302] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, 5affbf94e5a04a4897570b1e2b73dbb6; no voters:
I20260808 22:55:23.007053 10640 raft_consensus.cc:2781] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:23.007113 10640 raft_consensus.cc:482] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:23.007139 10640 raft_consensus.cc:3037] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.007898 10640 raft_consensus.cc:504] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:23.008066 10640 leader_election.cc:288] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 election: Requested vote from peers ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.008280 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4a4dfcc215e439a81a67e85c47c4c12" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb"
I20260808 22:55:23.008270 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4a4dfcc215e439a81a67e85c47c4c12" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
I20260808 22:55:23.008365 10398 raft_consensus.cc:3037] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.008365 10552 raft_consensus.cc:3037] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.009338 10552 raft_consensus.cc:2454] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 1.
I20260808 22:55:23.009334 10398 raft_consensus.cc:2454] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 1.
I20260808 22:55:23.009560 10174 leader_election.cc:302] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, 5affbf94e5a04a4897570b1e2b73dbb6; no voters:
I20260808 22:55:23.009708 10640 raft_consensus.cc:2781] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:23.009933 10640 raft_consensus.cc:686] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 LEADER]: Becoming Leader. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Running, Role: LEADER
I20260808 22:55:23.010064 10640 consensus_queue.cc:237] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } }
I20260808 22:55:23.010998 10072 catalog_manager.cc:5495] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "4d8994928dd545528e2f6f2441c8c4c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } health_report { overall_health: UNKNOWN } } }
I20260808 22:55:23.033210 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.030663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.1.212.1:38085)
0808 22:55:23.030744 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.030746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.030753 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:55:23.030792 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.030793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.030815 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.030895 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:23.031045 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.031548 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.032348 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.032558 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.032639 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.032831 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.032952 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.033061 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.033086 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.033111 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:55:23.035229 10552 raft_consensus.cc:1261] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Refusing update from remote peer 4d8994928dd545528e2f6f2441c8c4c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.035229 10398 raft_consensus.cc:1261] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Refusing update from remote peer 4d8994928dd545528e2f6f2441c8c4c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.035661 10644 consensus_queue.cc:1035] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:55:23.035744 10603 tablet.cc:2369] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:23.035838 10640 consensus_queue.cc:1035] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.038955 10649 mvcc.cc:205] Tried to move back new op lower bound from 7316396945549438976 to 7316396945450041344. Current Snapshot: MvccSnapshot[applied={T|T < 7316396945549438976}]
I20260808 22:55:23.039214 10650 mvcc.cc:205] Tried to move back new op lower bound from 7316396945549438976 to 7316396945450041344. Current Snapshot: MvccSnapshot[applied={T|T < 7316396945549438976}]
I20260808 22:55:23.041083 10072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.041199 10072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.042011 10072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Sending DeleteTablet for 3 replicas of tablet a4a4dfcc215e439a81a67e85c47c4c12
I20260808 22:55:23.042351 10530 tablet_service.cc:1514] Processing DeleteTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43140
I20260808 22:55:23.042397 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43086
I20260808 22:55:23.042528 10376 tablet_service.cc:1514] Processing DeleteTablet for tablet a4a4dfcc215e439a81a67e85c47c4c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:60310
I20260808 22:55:23.042644 10659 tablet_replica.cc:330] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: stopping tablet replica
I20260808 22:55:23.042743 10659 raft_consensus.cc:2229] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:23.042867 10659 raft_consensus.cc:2258] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.043085 10658 tablet_replica.cc:330] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: stopping tablet replica
I20260808 22:55:23.043026 10072 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:23.043180 10072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:23.043315 10658 raft_consensus.cc:2229] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.043418 10659 ts_tablet_manager.cc:1905] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.043478 10658 raft_consensus.cc:2258] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.043938 10658 ts_tablet_manager.cc:1905] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.044696 10224 tablet_service.cc:1467] Processing CreateTablet for tablet bc94ae744ad446bb9f725a032ce332d7 (DEFAULT_TABLE table=test-table [id=b5133f57a7a44c488933b201e0ddf9b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.044859 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc94ae744ad446bb9f725a032ce332d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.045555 10661 tablet_replica.cc:330] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: stopping tablet replica
I20260808 22:55:23.045715 10661 raft_consensus.cc:2229] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.045897 10661 raft_consensus.cc:2258] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.045934 10376 tablet_service.cc:1467] Processing CreateTablet for tablet bc94ae744ad446bb9f725a032ce332d7 (DEFAULT_TABLE table=test-table [id=b5133f57a7a44c488933b201e0ddf9b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.046145 10659 ts_tablet_manager.cc:1918] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.046173 10376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc94ae744ad446bb9f725a032ce332d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.046240 10659 log.cc:1196] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/a4a4dfcc215e439a81a67e85c47c4c12
I20260808 22:55:23.046191 10658 ts_tablet_manager.cc:1918] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.046320 10658 log.cc:1196] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/a4a4dfcc215e439a81a67e85c47c4c12
I20260808 22:55:23.046555 10659 ts_tablet_manager.cc:1939] T a4a4dfcc215e439a81a67e85c47c4c12 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting consensus metadata
I20260808 22:55:23.046592 10658 ts_tablet_manager.cc:1939] T a4a4dfcc215e439a81a67e85c47c4c12 P ede8c626dc604d2d9acb3bee5c168deb: Deleting consensus metadata
I20260808 22:55:23.047627 10637 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Bootstrap starting.
I20260808 22:55:23.047842 10054 catalog_manager.cc:4841] TS ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643): tablet a4a4dfcc215e439a81a67e85c47c4c12 (table test-table [id=6bdea425a18c4841bf2bdfff52020d00]) successfully deleted
I20260808 22:55:23.047978 10057 catalog_manager.cc:4841] TS 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085): tablet a4a4dfcc215e439a81a67e85c47c4c12 (table test-table [id=6bdea425a18c4841bf2bdfff52020d00]) successfully deleted
I20260808 22:55:23.048152 10637 tablet_bootstrap.cc:654] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.048224 10530 tablet_service.cc:1467] Processing CreateTablet for tablet bc94ae744ad446bb9f725a032ce332d7 (DEFAULT_TABLE table=test-table [id=b5133f57a7a44c488933b201e0ddf9b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.048347 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc94ae744ad446bb9f725a032ce332d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.048892 10637 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: No bootstrap required, opened a new log
I20260808 22:55:23.048951 10637 ts_tablet_manager.cc:1397] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:23.049131 10637 raft_consensus.cc:348] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.049218 10637 raft_consensus.cc:374] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.049250 10637 raft_consensus.cc:729] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.049305 10661 ts_tablet_manager.cc:1905] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.049304 10637 consensus_queue.cc:260] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.049448 10637 ts_tablet_manager.cc:1428] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:55:23.049746 10632 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Bootstrap starting.
I20260808 22:55:23.050169 10632 tablet_bootstrap.cc:654] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.051122 10661 ts_tablet_manager.cc:1918] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.051209 10661 log.cc:1196] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/a4a4dfcc215e439a81a67e85c47c4c12
I20260808 22:55:23.051292 10632 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: No bootstrap required, opened a new log
I20260808 22:55:23.051347 10632 ts_tablet_manager.cc:1397] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:55:23.051498 10661 ts_tablet_manager.cc:1939] T a4a4dfcc215e439a81a67e85c47c4c12 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting consensus metadata
I20260808 22:55:23.051523 10632 raft_consensus.cc:348] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.051604 10632 raft_consensus.cc:374] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.051632 10632 raft_consensus.cc:729] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.051682 10632 consensus_queue.cc:260] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.051777 10632 ts_tablet_manager.cc:1428] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.051990 10057 catalog_manager.cc:4841] TS 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149): tablet a4a4dfcc215e439a81a67e85c47c4c12 (table test-table [id=6bdea425a18c4841bf2bdfff52020d00]) successfully deleted
I20260808 22:55:23.052897 10633 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Bootstrap starting.
I20260808 22:55:23.053310 10633 tablet_bootstrap.cc:654] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.054013 10633 tablet_bootstrap.cc:492] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: No bootstrap required, opened a new log
I20260808 22:55:23.054083 10633 ts_tablet_manager.cc:1397] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:23.054245 10633 raft_consensus.cc:348] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.054327 10633 raft_consensus.cc:374] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.054352 10633 raft_consensus.cc:729] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.054406 10633 consensus_queue.cc:260] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.054502 10633 ts_tablet_manager.cc:1428] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.345465 10654 raft_consensus.cc:482] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:23.345590 10654 raft_consensus.cc:504] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.345829 10654 leader_election.cc:288] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149), ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:23.346124 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc94ae744ad446bb9f725a032ce332d7" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" is_pre_election: true
I20260808 22:55:23.346097 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc94ae744ad446bb9f725a032ce332d7" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb" is_pre_election: true
I20260808 22:55:23.346233 10398 raft_consensus.cc:2454] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 0.
I20260808 22:55:23.346231 10552 raft_consensus.cc:2454] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 0.
I20260808 22:55:23.346423 10170 leader_election.cc:302] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:23.346544 10654 raft_consensus.cc:2781] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:23.346580 10654 raft_consensus.cc:482] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:23.346606 10654 raft_consensus.cc:3037] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.347576 10654 raft_consensus.cc:504] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.347756 10654 leader_election.cc:288] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 election: Requested vote from peers 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149), ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:23.347942 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc94ae744ad446bb9f725a032ce332d7" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
I20260808 22:55:23.347980 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc94ae744ad446bb9f725a032ce332d7" candidate_uuid: "4d8994928dd545528e2f6f2441c8c4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb"
I20260808 22:55:23.348054 10398 raft_consensus.cc:3037] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.348106 10552 raft_consensus.cc:3037] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.348831 10398 raft_consensus.cc:2454] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 1.
I20260808 22:55:23.348932 10552 raft_consensus.cc:2454] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8994928dd545528e2f6f2441c8c4c0 in term 1.
I20260808 22:55:23.349026 10174 leader_election.cc:302] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, 5affbf94e5a04a4897570b1e2b73dbb6; no voters:
I20260808 22:55:23.349124 10654 raft_consensus.cc:2781] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:23.349176 10654 raft_consensus.cc:686] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 LEADER]: Becoming Leader. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Running, Role: LEADER
I20260808 22:55:23.349231 10654 consensus_queue.cc:237] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.349861 10072 catalog_manager.cc:5495] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "4d8994928dd545528e2f6f2441c8c4c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:55:23 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229722, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.62@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:23.437515 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.429693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.1.212.62:40221)
0808 22:55:23.429847 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.429849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.429859 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:23.429913 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.429913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.429925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.429999 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:23.430513 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.431023 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.431843 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.432052 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.432111 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.434538 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:23.434540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:23.434540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:23.434549 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:23.436107 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:23.436558 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:23.436559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:23.436560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:23.436635 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:23.436870 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:23.436871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:23.436872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:23.437000 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:23.437087 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.437252 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.437366 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":101,"threads_started":1}
I20260808 22:55:23.437537 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.429603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40221 (local address 127.0.0.1:53076)
0808 22:55:23.429789 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.429800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.429853 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.430019 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.430020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.430326 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:23.430377 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.430380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.431134 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.431135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.431754 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.431756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.432127 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.432128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.432190 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.432745 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:23.432751 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:23.434436 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:23.436224 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:23.436236 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:23.436239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:23.436466 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:23.436715 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:23.436716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:23.436716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:23.436819 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:23.437074 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:23.437085 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:23.437136 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.437226 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.437429 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":83,"thread_start_us":105,"threads_started":1}
I20260808 22:55:23.439765 10072 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:53076:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:55:23.440026 10072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:23.441679 10224 tablet_service.cc:1467] Processing CreateTablet for tablet fc430740ccd2497fb94fdae115fc62d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c765086bc904ddc8b3be8c183dcd534]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:55:23.441679 10530 tablet_service.cc:1467] Processing CreateTablet for tablet fc430740ccd2497fb94fdae115fc62d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c765086bc904ddc8b3be8c183dcd534]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:55:23.441730 10376 tablet_service.cc:1467] Processing CreateTablet for tablet fc430740ccd2497fb94fdae115fc62d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c765086bc904ddc8b3be8c183dcd534]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:55:23.441857 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc430740ccd2497fb94fdae115fc62d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.441857 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc430740ccd2497fb94fdae115fc62d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.441892 10376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc430740ccd2497fb94fdae115fc62d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.443492 10637 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: Bootstrap starting.
I20260808 22:55:23.443542 10632 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: Bootstrap starting.
I20260808 22:55:23.443959 10632 tablet_bootstrap.cc:654] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.443950 10637 tablet_bootstrap.cc:654] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.444362 10633 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: Bootstrap starting.
I20260808 22:55:23.444669 10637 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: No bootstrap required, opened a new log
I20260808 22:55:23.444708 10633 tablet_bootstrap.cc:654] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.444732 10637 ts_tablet_manager.cc:1397] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:23.444943 10632 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: No bootstrap required, opened a new log
I20260808 22:55:23.444990 10632 ts_tablet_manager.cc:1397] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:23.444928 10637 raft_consensus.cc:348] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445027 10637 raft_consensus.cc:374] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.445052 10637 raft_consensus.cc:729] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.445111 10637 consensus_queue.cc:260] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445163 10632 raft_consensus.cc:348] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445230 10637 ts_tablet_manager.cc:1428] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:55:23.445245 10632 raft_consensus.cc:374] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.445271 10632 raft_consensus.cc:729] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.445297 10641 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.445328 10632 consensus_queue.cc:260] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445351 10633 tablet_bootstrap.cc:492] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: No bootstrap required, opened a new log
I20260808 22:55:23.445371 10641 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.445400 10633 ts_tablet_manager.cc:1397] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:23.445472 10632 ts_tablet_manager.cc:1428] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.445511 10633 raft_consensus.cc:348] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445580 10633 raft_consensus.cc:374] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.445604 10633 raft_consensus.cc:729] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.445653 10633 consensus_queue.cc:260] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.445731 10633 ts_tablet_manager.cc:1428] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.445870 10639 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.445971 10639 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.446094 10654 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.446183 10654 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.458664 10639 raft_consensus.cc:482] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:23.458804 10639 raft_consensus.cc:504] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.459200 10639 leader_election.cc:288] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.462704 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.459536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:42583 (local address 127.1.212.1:38085)
0808 22:55:23.459624 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.459626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.459679 (+ 53us) server_negotiation.cc:407] Connection header received
0808 22:55:23.459781 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.459783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.459793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.459828 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.460082 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.460586 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.461955 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.462317 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.462374 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.462383 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.462577 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.462626 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:55:23.462710 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.459732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:49059 (local address 127.1.212.2:46149)
0808 22:55:23.459804 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.459806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.459815 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:23.459870 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.459871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.459880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.459950 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.460122 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.460599 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.461911 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.462326 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.462439 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.462457 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.462606 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.462644 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:55:23.462740 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.459643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:46149 (local address 127.1.212.3:49059)
0808 22:55:23.459735 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.459745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.459800 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.459954 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.459955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.459957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.460054 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.460056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.460700 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.460701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.461818 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.461822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.462442 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.462443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.462507 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.462510 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.462591 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.462632 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":57,"thread_start_us":55,"threads_started":1}
I20260808 22:55:23.462734 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.459443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38085 (local address 127.1.212.3:42583)
0808 22:55:23.459573 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.459588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.459682 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.459918 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.459918 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.459920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.459998 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.460000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.460678 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.460679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.461833 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.461837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.462407 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.462409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.462489 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.462493 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.462547 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.462611 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":19,"thread_start_us":74,"threads_started":1}
I20260808 22:55:23.463274 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc430740ccd2497fb94fdae115fc62d4" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0" is_pre_election: true
I20260808 22:55:23.463447 10244 raft_consensus.cc:2454] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 0.
I20260808 22:55:23.463580 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc430740ccd2497fb94fdae115fc62d4" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" is_pre_election: true
I20260808 22:55:23.463675 10398 raft_consensus.cc:2454] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 0.
I20260808 22:55:23.463639 10481 leader_election.cc:302] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:23.463833 10639 raft_consensus.cc:2781] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:23.463883 10639 raft_consensus.cc:482] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:23.463898 10639 raft_consensus.cc:3037] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.464651 10639 raft_consensus.cc:504] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.464838 10639 leader_election.cc:288] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 election: Requested vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.465050 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc430740ccd2497fb94fdae115fc62d4" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
I20260808 22:55:23.465149 10398 raft_consensus.cc:3037] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.465871 10398 raft_consensus.cc:2454] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 1.
I20260808 22:55:23.466037 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc430740ccd2497fb94fdae115fc62d4" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0"
I20260808 22:55:23.466090 10481 leader_election.cc:302] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5affbf94e5a04a4897570b1e2b73dbb6, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:23.466125 10244 raft_consensus.cc:3037] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.466279 10639 raft_consensus.cc:2781] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:23.466473 10639 raft_consensus.cc:686] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [term 1 LEADER]: Becoming Leader. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Running, Role: LEADER
I20260808 22:55:23.466560 10639 consensus_queue.cc:237] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.466995 10682 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: TxnStatusTablet state changed. Reason: New leader ede8c626dc604d2d9acb3bee5c168deb. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.467060 10244 raft_consensus.cc:2454] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 1.
I20260808 22:55:23.467092 10682 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:55:23.467350 10069 catalog_manager.cc:5495] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb reported cstate change: term changed from 0 to 1, leader changed from <none> to ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3). New cstate: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } health_report { overall_health: HEALTHY } } }
I20260808 22:55:23.467641 10684 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:55:23.467758 10684 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:55:23.478935 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.475806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.1.212.3:36643)
0808 22:55:23.475880 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.475882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.475967 (+ 85us) server_negotiation.cc:407] Connection header received
0808 22:55:23.476066 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.476067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.476087 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.476152 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:23.476350 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.476958 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.478121 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.478316 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.478375 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.478595 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.478706 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.478763 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.478847 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.478869 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:55:23.478991 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.475730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.0.0.1:43144)
0808 22:55:23.475897 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.475911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.475977 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.476173 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.476174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.476175 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:55:23.476236 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.476239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.477058 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.477059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.478011 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.478015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.478404 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.478405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.478471 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.478495 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.478778 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.478787 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.478843 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.478904 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260808 22:55:23.480376 10398 raft_consensus.cc:1261] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Refusing update from remote peer ede8c626dc604d2d9acb3bee5c168deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.480460 10244 raft_consensus.cc:1261] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Refusing update from remote peer ede8c626dc604d2d9acb3bee5c168deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.480731 10639 consensus_queue.cc:1035] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.480901 10682 consensus_queue.cc:1035] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.481748 10654 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: TxnStatusTablet state changed. Reason: New leader ede8c626dc604d2d9acb3bee5c168deb. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.481855 10654 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.482013 10641 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: TxnStatusTablet state changed. Reason: New leader ede8c626dc604d2d9acb3bee5c168deb. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.482120 10641 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.482678 10682 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.482798 10682 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:55:23.483174 10682 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.483256 10682 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P ede8c626dc604d2d9acb3bee5c168deb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:55:23.483253 10641 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.483346 10641 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 5affbf94e5a04a4897570b1e2b73dbb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.483816 10648 mvcc.cc:205] Tried to move back new op lower bound from 7316396947373117440 to 7316396947319582720. Current Snapshot: MvccSnapshot[applied={T|T < 7316396947373117440 or (T in {7316396947373117440})}]
I20260808 22:55:23.483942 10654 tablet_replica.cc:439] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } } }
I20260808 22:55:23.484030 10654 tablet_replica.cc:442] T fc430740ccd2497fb94fdae115fc62d4 P 4d8994928dd545528e2f6f2441c8c4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:55:23.489552 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.486814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:43759 (local address 127.1.212.3:36643)
0808 22:55:23.486867 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.486869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.487000 (+ 131us) server_negotiation.cc:407] Connection header received
0808 22:55:23.487079 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.487080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.487092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.487132 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:23.487274 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.487757 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.488689 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.488969 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.489046 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.489241 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.489278 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.489347 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.489433 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.489455 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:55:23.489555 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.486744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.1.212.1:43759)
0808 22:55:23.486895 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.486932 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.487017 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.487142 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.487143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.487145 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:55:23.487203 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.487205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.487852 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.487853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.488600 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.488603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.489046 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.489047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.489109 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.489138 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.489349 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.489352 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.489428 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.489481 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260808 22:55:23.492321 10552 raft_consensus.cc:1261] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Refusing update from remote peer 4d8994928dd545528e2f6f2441c8c4c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.492331 10398 raft_consensus.cc:1261] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Refusing update from remote peer 4d8994928dd545528e2f6f2441c8c4c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.492558 10654 consensus_queue.cc:1035] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.492681 10654 consensus_queue.cc:1035] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.502209 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.499364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:47935 (local address 127.1.212.1:38085)
0808 22:55:23.499420 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.499423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.499571 (+ 148us) server_negotiation.cc:407] Connection header received
0808 22:55:23.499644 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.499645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.499656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.499701 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:23.499845 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.500317 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.501520 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.501722 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.501776 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.501992 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.502022 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.502060 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.502124 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.502145 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:55:23.502281 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.499314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38085 (local address 127.1.212.3:47935)
0808 22:55:23.499516 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.499527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.499586 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.499705 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.499706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.499707 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:55:23.499786 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.499788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.500403 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.500404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.501442 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.501445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.501801 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.501802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.501883 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.501906 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.502069 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.502070 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.502123 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.502171 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260808 22:55:23.511541 10069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.511678 10069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.512413 10069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Sending DeleteTablet for 3 replicas of tablet bc94ae744ad446bb9f725a032ce332d7
I20260808 22:55:23.512708 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet bc94ae744ad446bb9f725a032ce332d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43086
I20260808 22:55:23.512730 10530 tablet_service.cc:1514] Processing DeleteTablet for tablet bc94ae744ad446bb9f725a032ce332d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43140
I20260808 22:55:23.512744 10376 tablet_service.cc:1514] Processing DeleteTablet for tablet bc94ae744ad446bb9f725a032ce332d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:60310
I20260808 22:55:23.512843 10659 tablet_replica.cc:330] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: stopping tablet replica
I20260808 22:55:23.512876 10658 tablet_replica.cc:330] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: stopping tablet replica
I20260808 22:55:23.512913 10659 raft_consensus.cc:2229] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:23.512935 10658 raft_consensus.cc:2229] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.512982 10658 raft_consensus.cc:2258] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.512987 10659 raft_consensus.cc:2258] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.513067 10661 tablet_replica.cc:330] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: stopping tablet replica
I20260808 22:55:23.513139 10661 raft_consensus.cc:2229] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.513187 10661 raft_consensus.cc:2258] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.513378 10658 ts_tablet_manager.cc:1905] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.513649 10659 ts_tablet_manager.cc:1905] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.515203 10658 ts_tablet_manager.cc:1918] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:55:23.515285 10658 log.cc:1196] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/bc94ae744ad446bb9f725a032ce332d7
I20260808 22:55:23.515389 10659 ts_tablet_manager.cc:1918] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:55:23.515455 10659 log.cc:1196] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/bc94ae744ad446bb9f725a032ce332d7
I20260808 22:55:23.515525 10658 ts_tablet_manager.cc:1939] T bc94ae744ad446bb9f725a032ce332d7 P ede8c626dc604d2d9acb3bee5c168deb: Deleting consensus metadata
I20260808 22:55:23.515686 10659 ts_tablet_manager.cc:1939] T bc94ae744ad446bb9f725a032ce332d7 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting consensus metadata
I20260808 22:55:23.515939 10054 catalog_manager.cc:4841] TS ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643): tablet bc94ae744ad446bb9f725a032ce332d7 (table test-table [id=b5133f57a7a44c488933b201e0ddf9b2]) successfully deleted
I20260808 22:55:23.516002 10661 ts_tablet_manager.cc:1905] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.517001 10057 catalog_manager.cc:4841] TS 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085): tablet bc94ae744ad446bb9f725a032ce332d7 (table test-table [id=b5133f57a7a44c488933b201e0ddf9b2]) successfully deleted
I20260808 22:55:23.517767 10661 ts_tablet_manager.cc:1918] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:55:23.517845 10661 log.cc:1196] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/bc94ae744ad446bb9f725a032ce332d7
I20260808 22:55:23.518080 10661 ts_tablet_manager.cc:1939] T bc94ae744ad446bb9f725a032ce332d7 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting consensus metadata
I20260808 22:55:23.518733 10057 catalog_manager.cc:4841] TS 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149): tablet bc94ae744ad446bb9f725a032ce332d7 (table test-table [id=b5133f57a7a44c488933b201e0ddf9b2]) successfully deleted
Aug 08 22:55:23 dist-test-slave-1x4g krb5kdc[10020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:55:23 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:23 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:55:23.545825 10675 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:23.538987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.1.212.62:40221)
0808 22:55:23.539055 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.539057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.539069 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:55:23.539148 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.539149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.539161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.539226 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:23.540027 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.540554 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.541715 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.541920 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.541996 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.545749 (+ 3753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53080: BlockingRecv error: recv got EOF from 127.0.0.1:53080 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 22:55:23 dist-test-slave-1x4g krb5kdc[10020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM
I20260808 22:55:23.554644 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.546750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.1.212.62:40221)
0808 22:55:23.546813 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.546814 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.546823 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:23.546862 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.546863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.546879 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.546920 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:23.547549 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.548090 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.548921 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.549127 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.549213 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.551674 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:23.551675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:23.551676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:23.551687 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:23.553355 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:23.553819 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:23.553821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:23.553821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:23.553888 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:23.554109 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:23.554110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:23.554111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:23.554263 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:23.554370 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.554464 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.554549 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:55:23.556335 10069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:23.556536 10069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:23.558241 10530 tablet_service.cc:1467] Processing CreateTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (DEFAULT_TABLE table=test-table [id=b94154c6225f4699b8ac60c7086df4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.558243 10224 tablet_service.cc:1467] Processing CreateTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (DEFAULT_TABLE table=test-table [id=b94154c6225f4699b8ac60c7086df4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.558321 10376 tablet_service.cc:1467] Processing CreateTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (DEFAULT_TABLE table=test-table [id=b94154c6225f4699b8ac60c7086df4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.558411 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f4dd992074495bbfe6cf70fcd8cfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.558411 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f4dd992074495bbfe6cf70fcd8cfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.558437 10376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f4dd992074495bbfe6cf70fcd8cfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.559849 10637 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Bootstrap starting.
I20260808 22:55:23.559849 10633 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Bootstrap starting.
I20260808 22:55:23.559880 10632 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Bootstrap starting.
I20260808 22:55:23.560226 10632 tablet_bootstrap.cc:654] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.560235 10633 tablet_bootstrap.cc:654] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.560288 10637 tablet_bootstrap.cc:654] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.561004 10632 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: No bootstrap required, opened a new log
I20260808 22:55:23.561057 10632 ts_tablet_manager.cc:1397] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:23.561069 10637 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: No bootstrap required, opened a new log
I20260808 22:55:23.561117 10637 ts_tablet_manager.cc:1397] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:23.561192 10632 raft_consensus.cc:348] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.561290 10632 raft_consensus.cc:374] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.561321 10632 raft_consensus.cc:729] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.561292 10637 raft_consensus.cc:348] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.561376 10637 raft_consensus.cc:374] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.561368 10632 consensus_queue.cc:260] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.561406 10637 raft_consensus.cc:729] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.561462 10632 ts_tablet_manager.cc:1428] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.561456 10637 consensus_queue.cc:260] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.561551 10637 ts_tablet_manager.cc:1428] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.561856 10633 tablet_bootstrap.cc:492] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: No bootstrap required, opened a new log
I20260808 22:55:23.561913 10633 ts_tablet_manager.cc:1397] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:23.562043 10633 raft_consensus.cc:348] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.562119 10633 raft_consensus.cc:374] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.562143 10633 raft_consensus.cc:729] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.562192 10633 consensus_queue.cc:260] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.562294 10633 ts_tablet_manager.cc:1428] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.786463 10641 raft_consensus.cc:482] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:23.786590 10641 raft_consensus.cc:504] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.787034 10641 leader_election.cc:288] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:23.788652 10682 raft_consensus.cc:482] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:23.788741 10682 raft_consensus.cc:504] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.788959 10682 leader_election.cc:288] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.789316 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" is_pre_election: true
I20260808 22:55:23.789314 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0" is_pre_election: true
I20260808 22:55:23.789443 10244 raft_consensus.cc:2454] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 0.
I20260808 22:55:23.789435 10398 raft_consensus.cc:2454] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 0.
I20260808 22:55:23.789800 10481 leader_election.cc:302] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5affbf94e5a04a4897570b1e2b73dbb6, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:23.789956 10682 raft_consensus.cc:2781] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:23.790015 10682 raft_consensus.cc:482] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:23.790059 10682 raft_consensus.cc:3037] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.791116 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.787479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:36433 (local address 127.1.212.3:36643)
0808 22:55:23.787531 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.787533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.787541 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:55:23.787570 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.787571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.787580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.787646 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.788003 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.788647 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.790293 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.790718 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.790813 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.790827 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.791000 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.791040 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:55:23.791153 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.787239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.1.212.2:36433)
0808 22:55:23.787393 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.787407 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.787484 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.787805 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.787806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.787807 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.787865 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.787866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.788828 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.788830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.790195 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.790199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.790827 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.790828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.790905 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.790910 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.790992 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.791039 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":33,"thread_start_us":78,"threads_started":1}
I20260808 22:55:23.791122 10682 raft_consensus.cc:504] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.791291 10682 leader_election.cc:288] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 election: Requested vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149)
I20260808 22:55:23.791424 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.787318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:44395 (local address 127.1.212.1:38085)
0808 22:55:23.787438 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.787440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.787525 (+ 85us) server_negotiation.cc:407] Connection header received
0808 22:55:23.787651 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.787652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.787675 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.787752 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:23.787927 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.788521 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.790404 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.790932 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.791011 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.791023 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.791310 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.791366 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 22:55:23.791580 10398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5affbf94e5a04a4897570b1e2b73dbb6"
I20260808 22:55:23.791630 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "ede8c626dc604d2d9acb3bee5c168deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0"
I20260808 22:55:23.791584 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb" is_pre_election: true
I20260808 22:55:23.791689 10244 raft_consensus.cc:3037] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.791689 10398 raft_consensus.cc:3037] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:23.791718 10552 raft_consensus.cc:2379] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5affbf94e5a04a4897570b1e2b73dbb6 in current term 1: Already voted for candidate ede8c626dc604d2d9acb3bee5c168deb in this term.
I20260808 22:55:23.791781 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.787241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38085 (local address 127.1.212.2:44395)
0808 22:55:23.787459 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:23.787470 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:23.787553 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:23.787757 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:23.787759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:23.787761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:23.787828 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.787831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.788674 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.788676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.790231 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:23.790235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.791062 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.791063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:23.791139 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.791144 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:23.791519 (+ 375us) client_negotiation.cc:240] Negotiation successful
0808 22:55:23.791574 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":52,"thread_start_us":73,"threads_started":1}
I20260808 22:55:23.792462 10398 raft_consensus.cc:2454] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 1.
I20260808 22:55:23.792462 10244 raft_consensus.cc:2454] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ede8c626dc604d2d9acb3bee5c168deb in term 1.
I20260808 22:55:23.792654 10481 leader_election.cc:302] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5affbf94e5a04a4897570b1e2b73dbb6, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:23.792690 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f4dd992074495bbfe6cf70fcd8cfc3" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0" is_pre_election: true
I20260808 22:55:23.792797 10682 raft_consensus.cc:2781] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:23.792814 10244 raft_consensus.cc:2379] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5affbf94e5a04a4897570b1e2b73dbb6 in current term 1: Already voted for candidate ede8c626dc604d2d9acb3bee5c168deb in this term.
I20260808 22:55:23.792888 10682 raft_consensus.cc:686] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 LEADER]: Becoming Leader. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Running, Role: LEADER
I20260808 22:55:23.792951 10682 consensus_queue.cc:237] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.792979 10323 leader_election.cc:302] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5affbf94e5a04a4897570b1e2b73dbb6; no voters: 4d8994928dd545528e2f6f2441c8c4c0, ede8c626dc604d2d9acb3bee5c168deb
I20260808 22:55:23.793183 10641 raft_consensus.cc:2726] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 22:55:23.793689 10072 catalog_manager.cc:5495] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb reported cstate change: term changed from 0 to 1, leader changed from <none> to ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3). New cstate: current_term: 1 leader_uuid: "ede8c626dc604d2d9acb3bee5c168deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } health_report { overall_health: HEALTHY } } }
I20260808 22:55:23.850447 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:23.848014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.1.212.3:36643)
0808 22:55:23.848096 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:23.848099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:23.848108 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:23.848135 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:23.848136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:23.848147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:23.848194 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:23.848352 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.848876 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.849711 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:23.849922 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:23.849980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:23.850169 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:23.850200 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:23.850244 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:23.850305 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 22:55:23.850325 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:55:23.851532 10398 raft_consensus.cc:1261] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Refusing update from remote peer ede8c626dc604d2d9acb3bee5c168deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.851550 10244 raft_consensus.cc:1261] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Refusing update from remote peer ede8c626dc604d2d9acb3bee5c168deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:23.851780 10682 consensus_queue.cc:1035] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.851917 10639 consensus_queue.cc:1035] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:23.855667 10069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.855785 10069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:23.856473 10069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Sending DeleteTablet for 3 replicas of tablet 63f4dd992074495bbfe6cf70fcd8cfc3
I20260808 22:55:23.856820 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43086
I20260808 22:55:23.856949 10376 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:60310
I20260808 22:55:23.857057 10659 tablet_replica.cc:330] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: stopping tablet replica
I20260808 22:55:23.857075 10661 tablet_replica.cc:330] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: stopping tablet replica
I20260808 22:55:23.857122 10659 raft_consensus.cc:2229] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.857161 10661 raft_consensus.cc:2229] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:23.857184 10659 raft_consensus.cc:2258] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.857230 10661 raft_consensus.cc:2258] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.857420 10530 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f4dd992074495bbfe6cf70fcd8cfc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:23 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43140
I20260808 22:55:23.857388 10069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:55:23.857523 10658 tablet_replica.cc:330] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: stopping tablet replica
W20260808 22:55:23.857554 10069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:23.857570 10658 raft_consensus.cc:2229] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:23.857641 10661 ts_tablet_manager.cc:1905] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.857673 10658 raft_consensus.cc:2258] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:23.857986 10658 ts_tablet_manager.cc:1905] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.859243 10376 tablet_service.cc:1467] Processing CreateTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 (DEFAULT_TABLE table=test-table [id=b42efb21afe04f2fb9542d1eeceb1bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.859462 10224 tablet_service.cc:1467] Processing CreateTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 (DEFAULT_TABLE table=test-table [id=b42efb21afe04f2fb9542d1eeceb1bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.859565 10658 ts_tablet_manager.cc:1918] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.859588 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3684abbae94dfaae2c3b4451b7bdc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.859596 10661 ts_tablet_manager.cc:1918] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.859625 10658 log.cc:1196] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/63f4dd992074495bbfe6cf70fcd8cfc3
I20260808 22:55:23.859643 10661 log.cc:1196] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/63f4dd992074495bbfe6cf70fcd8cfc3
I20260808 22:55:23.859858 10658 ts_tablet_manager.cc:1939] T 63f4dd992074495bbfe6cf70fcd8cfc3 P ede8c626dc604d2d9acb3bee5c168deb: Deleting consensus metadata
I20260808 22:55:23.859871 10661 ts_tablet_manager.cc:1939] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting consensus metadata
I20260808 22:55:23.860035 10376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3684abbae94dfaae2c3b4451b7bdc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.861117 10633 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: Bootstrap starting.
I20260808 22:55:23.861356 10637 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Bootstrap starting.
I20260808 22:55:23.861452 10633 tablet_bootstrap.cc:654] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.861799 10637 tablet_bootstrap.cc:654] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.861783 10054 catalog_manager.cc:4841] TS ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643): tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (table test-table [id=b94154c6225f4699b8ac60c7086df4e0]) successfully deleted
I20260808 22:55:23.861820 10530 tablet_service.cc:1467] Processing CreateTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 (DEFAULT_TABLE table=test-table [id=b42efb21afe04f2fb9542d1eeceb1bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:23.861927 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3684abbae94dfaae2c3b4451b7bdc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:23.862234 10659 ts_tablet_manager.cc:1905] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:23.862679 10057 catalog_manager.cc:4841] TS 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149): tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (table test-table [id=b94154c6225f4699b8ac60c7086df4e0]) successfully deleted
I20260808 22:55:23.863181 10632 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Bootstrap starting.
I20260808 22:55:23.863317 10633 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: No bootstrap required, opened a new log
I20260808 22:55:23.863363 10633 ts_tablet_manager.cc:1397] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:55:23.863525 10632 tablet_bootstrap.cc:654] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:23.863489 10633 raft_consensus.cc:348] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.863564 10633 raft_consensus.cc:374] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.863584 10633 raft_consensus.cc:729] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8994928dd545528e2f6f2441c8c4c0, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.863631 10633 consensus_queue.cc:260] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.863718 10633 ts_tablet_manager.cc:1428] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:55:23.863829 10659 ts_tablet_manager.cc:1918] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:23.863878 10659 log.cc:1196] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/63f4dd992074495bbfe6cf70fcd8cfc3
I20260808 22:55:23.864122 10659 ts_tablet_manager.cc:1939] T 63f4dd992074495bbfe6cf70fcd8cfc3 P 4d8994928dd545528e2f6f2441c8c4c0: Deleting consensus metadata
I20260808 22:55:23.864598 10057 catalog_manager.cc:4841] TS 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085): tablet 63f4dd992074495bbfe6cf70fcd8cfc3 (table test-table [id=b94154c6225f4699b8ac60c7086df4e0]) successfully deleted
I20260808 22:55:23.865005 10637 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: No bootstrap required, opened a new log
I20260808 22:55:23.865064 10637 ts_tablet_manager.cc:1397] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:55:23.865212 10637 raft_consensus.cc:348] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.865265 10637 raft_consensus.cc:374] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.865278 10637 raft_consensus.cc:729] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.865348 10637 consensus_queue.cc:260] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.865450 10637 ts_tablet_manager.cc:1428] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:23.865762 10632 tablet_bootstrap.cc:492] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: No bootstrap required, opened a new log
I20260808 22:55:23.865837 10632 ts_tablet_manager.cc:1397] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 22:55:23.865988 10632 raft_consensus.cc:348] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.866067 10632 raft_consensus.cc:374] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:23.866094 10632 raft_consensus.cc:729] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede8c626dc604d2d9acb3bee5c168deb, State: Initialized, Role: FOLLOWER
I20260808 22:55:23.866132 10632 consensus_queue.cc:260] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:23.866225 10632 ts_tablet_manager.cc:1428] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:24.089936 10641 raft_consensus.cc:482] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:24.090092 10641 raft_consensus.cc:504] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:24.090302 10641 leader_election.cc:288] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:24.090582 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3684abbae94dfaae2c3b4451b7bdc8" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb" is_pre_election: true
I20260808 22:55:24.090689 10552 raft_consensus.cc:2454] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5affbf94e5a04a4897570b1e2b73dbb6 in term 0.
I20260808 22:55:24.090591 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3684abbae94dfaae2c3b4451b7bdc8" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0" is_pre_election: true
I20260808 22:55:24.090739 10244 raft_consensus.cc:2454] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5affbf94e5a04a4897570b1e2b73dbb6 in term 0.
I20260808 22:55:24.090900 10320 leader_election.cc:302] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5affbf94e5a04a4897570b1e2b73dbb6, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:24.091061 10641 raft_consensus.cc:2781] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:24.091126 10641 raft_consensus.cc:482] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:24.091145 10641 raft_consensus.cc:3037] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:24.092010 10641 raft_consensus.cc:504] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:24.092195 10641 leader_election.cc:288] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 election: Requested vote from peers 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085), ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643)
I20260808 22:55:24.092406 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3684abbae94dfaae2c3b4451b7bdc8" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede8c626dc604d2d9acb3bee5c168deb"
I20260808 22:55:24.092494 10552 raft_consensus.cc:3037] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:24.092413 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3684abbae94dfaae2c3b4451b7bdc8" candidate_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8994928dd545528e2f6f2441c8c4c0"
I20260808 22:55:24.092526 10244 raft_consensus.cc:3037] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:24.093302 10552 raft_consensus.cc:2454] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5affbf94e5a04a4897570b1e2b73dbb6 in term 1.
I20260808 22:55:24.093321 10244 raft_consensus.cc:2454] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5affbf94e5a04a4897570b1e2b73dbb6 in term 1.
I20260808 22:55:24.093521 10323 leader_election.cc:302] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4d8994928dd545528e2f6f2441c8c4c0, 5affbf94e5a04a4897570b1e2b73dbb6, ede8c626dc604d2d9acb3bee5c168deb; no voters:
I20260808 22:55:24.093695 10641 raft_consensus.cc:2781] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:24.093854 10641 raft_consensus.cc:686] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 LEADER]: Becoming Leader. State: Replica: 5affbf94e5a04a4897570b1e2b73dbb6, State: Running, Role: LEADER
I20260808 22:55:24.093955 10641 consensus_queue.cc:237] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } }
I20260808 22:55:24.094738 10069 catalog_manager.cc:5495] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5affbf94e5a04a4897570b1e2b73dbb6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 46149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 } health_report { overall_health: UNKNOWN } } }
I20260808 22:55:24.157599 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.155057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.1.212.2:46149)
0808 22:55:24.155207 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:24.155210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:24.155219 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:24.155267 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:24.155267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:24.155290 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:24.155366 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:24.155475 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.155994 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.156716 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.157005 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.157059 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:24.157233 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:24.157348 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:24.157408 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:24.157444 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 22:55:24.157463 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260808 22:55:24.163151 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.160241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:36643 (local address 127.1.212.2:51811)
0808 22:55:24.160451 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:24.160462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:24.160557 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:24.160699 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:24.160700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:24.160702 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:55:24.160814 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:24.160816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.161453 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.161454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:24.162055 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:24.162057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.162605 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.162606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:24.162699 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:24.162719 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:55:24.162950 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:55:24.162953 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:24.163006 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:55:24.163058 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260808 22:55:24.163129 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.160346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51811 (local address 127.1.212.3:36643)
0808 22:55:24.160422 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:24.160425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:24.160538 (+ 113us) server_negotiation.cc:407] Connection header received
0808 22:55:24.160612 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:24.160614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:24.160626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:24.160695 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:24.160869 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.161361 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.162163 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.162487 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.162602 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:24.162813 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:24.162863 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:24.162942 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:24.163011 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:55:24.163043 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:55:24.165772 10552 raft_consensus.cc:1261] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Refusing update from remote peer 5affbf94e5a04a4897570b1e2b73dbb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:24.165982 10244 raft_consensus.cc:1261] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Refusing update from remote peer 5affbf94e5a04a4897570b1e2b73dbb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:24.166054 10641 consensus_queue.cc:1035] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede8c626dc604d2d9acb3bee5c168deb" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:24.166209 10726 consensus_queue.cc:1035] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8994928dd545528e2f6f2441c8c4c0" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:24.174518 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.171857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:53289 (local address 127.1.212.2:46149)
0808 22:55:24.171904 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:24.171906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:24.171993 (+ 87us) server_negotiation.cc:407] Connection header received
0808 22:55:24.172060 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:24.172062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:24.172075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:24.172133 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:24.172265 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.172822 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.173737 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.174035 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.174127 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:24.174297 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:24.174336 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:24.174372 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:24.174415 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 22:55:24.174438 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:55:24.174530 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.171791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:46149 (local address 127.1.212.3:53289)
0808 22:55:24.171925 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:24.171937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:24.171999 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:24.172137 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:24.172138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:24.172139 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:55:24.172194 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:24.172196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.172905 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.172907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:24.173555 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:24.173560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:24.174133 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:24.174135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:24.174221 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:24.174227 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:55:24.174380 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:55:24.174381 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:55:24.174414 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 22:55:24.174463 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260808 22:55:24.185256 10069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:24.185395 10069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:24.186161 10069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce25872566a4e31bd0660c01e41d0ad: Sending DeleteTablet for 3 replicas of tablet ad3684abbae94dfaae2c3b4451b7bdc8
I20260808 22:55:24.186451 10530 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:24 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43140
I20260808 22:55:24.186507 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:24 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43086
I20260808 22:55:24.186556 10658 tablet_replica.cc:330] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: stopping tablet replica
I20260808 22:55:24.186609 10659 tablet_replica.cc:330] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0: stopping tablet replica
I20260808 22:55:24.186617 10658 raft_consensus.cc:2229] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:24.186657 10658 raft_consensus.cc:2258] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:24.186666 10659 raft_consensus.cc:2229] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:24.186702 10659 raft_consensus.cc:2258] T ad3684abbae94dfaae2c3b4451b7bdc8 P 4d8994928dd545528e2f6f2441c8c4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:24.186822 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10152
I20260808 22:55:24.187244 10658 ts_tablet_manager.cc:1905] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:24.189553 10658 ts_tablet_manager.cc:1918] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:55:24.189633 10658 log.cc:1196] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/ad3684abbae94dfaae2c3b4451b7bdc8
I20260808 22:55:24.189913 10658 ts_tablet_manager.cc:1939] T ad3684abbae94dfaae2c3b4451b7bdc8 P ede8c626dc604d2d9acb3bee5c168deb: Deleting consensus metadata
I20260808 22:55:24.190358 10054 catalog_manager.cc:4841] TS ede8c626dc604d2d9acb3bee5c168deb (127.1.212.3:36643): tablet ad3684abbae94dfaae2c3b4451b7bdc8 (table test-table [id=b42efb21afe04f2fb9542d1eeceb1bc4]) successfully deleted
I20260808 22:55:24.190464 10376 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3684abbae94dfaae2c3b4451b7bdc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:24 UTC) from {username='oryx', principal='oryx/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:60310
I20260808 22:55:24.190618 10661 tablet_replica.cc:330] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: stopping tablet replica
I20260808 22:55:24.190697 10661 raft_consensus.cc:2229] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:24.190791 10661 raft_consensus.cc:2258] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:24.191157 10661 ts_tablet_manager.cc:1905] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:24.192631 10661 ts_tablet_manager.cc:1918] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:55:24.192706 10661 log.cc:1196] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/ad3684abbae94dfaae2c3b4451b7bdc8
I20260808 22:55:24.192984 10661 ts_tablet_manager.cc:1939] T ad3684abbae94dfaae2c3b4451b7bdc8 P 5affbf94e5a04a4897570b1e2b73dbb6: Deleting consensus metadata
I20260808 22:55:24.193382 10057 catalog_manager.cc:4841] TS 5affbf94e5a04a4897570b1e2b73dbb6 (127.1.212.2:46149): tablet ad3684abbae94dfaae2c3b4451b7bdc8 (table test-table [id=b42efb21afe04f2fb9542d1eeceb1bc4]) successfully deleted
W20260808 22:55:24.200393 10057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:38085 (error 108)
I20260808 22:55:24.200739 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10303
I20260808 22:55:24.203676 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:24.200892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:38085 (local address 127.0.0.1:43108)
0808 22:55:24.203524 (+ 2632us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:24.203607 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2602,"thread_start_us":65,"threads_started":1}
W20260808 22:55:24.203821 10057 catalog_manager.cc:4584] TS 4d8994928dd545528e2f6f2441c8c4c0 (127.1.212.1:38085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad3684abbae94dfaae2c3b4451b7bdc8: Network error: Client connection negotiation failed: client connection to 127.1.212.1:38085: connect: Connection refused (error 111)
I20260808 22:55:24.208998 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10458
I20260808 22:55:24.219082 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10037
2026-08-08T22:55:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3371 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:24 dist-test-slave-1x4g krb5kdc[10743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:24 dist-test-slave-1x4g krb5kdc[10743](info): set up 2 sockets
Aug 08 22:55:24 dist-test-slave-1x4g krb5kdc[10743](info): commencing operation
krb5kdc: starting...
W20260808 22:55:25.305601 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:25 dist-test-slave-1x4g krb5kdc[10743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:25.449122 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45823
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:45823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:25.539871 10760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:25.540169 10760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:25.540213 10760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:25.541599 10760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:25.541682 10760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:25.541707 10760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:25.541731 10760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:25.541761 10760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:25.543529 10760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:45823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:25.543851 10760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:25.544121 10760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:25.549363 10765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:25.549358 10766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:25.549363 10768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:25.549680 10760 server_base.cc:1034] running on GCE node
I20260808 22:55:25.549965 10760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:25.550289 10760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:25.551502 10760 hybrid_clock.cc:648] HybridClock initialized: now 1786229725551490 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T22:55:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1335 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:25 dist-test-slave-1x4g krb5kdc[10773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:25 dist-test-slave-1x4g krb5kdc[10773](info): set up 2 sockets
Aug 08 22:55:25 dist-test-slave-1x4g krb5kdc[10773](info): commencing operation
krb5kdc: starting...
W20260808 22:55:26.637166 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:26 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:26.776089 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37125
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:26.866596 10787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:26.866878 10787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:26.866931 10787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:26.868207 10787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:26.868287 10787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:26.868316 10787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:26.868351 10787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:26.868376 10787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:26.870132 10787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:26.870471 10787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:26.870733 10787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:26.876235 10793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:26.876235 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:26.876252 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:26.876518 10787 server_base.cc:1034] running on GCE node
I20260808 22:55:26.876830 10787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:26.877198 10787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:26.878392 10787 hybrid_clock.cc:648] HybridClock initialized: now 1786229726878375 us; error 49 us; skew 500 ppm
Aug 08 22:55:26 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229726, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:26.885555 10787 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:26.886147 10787 webserver.cc:460] Webserver started at http://127.1.212.62:35199/ using document root <none> and password file <none>
I20260808 22:55:26.886413 10787 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:26.886492 10787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:26.886655 10787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:26.887894 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "881456130cff477888b200c593107b56"
format_stamp: "Formatted at 2026-08-08 22:55:26 on dist-test-slave-1x4g"
server_key: "9f087b475e2db6928b1e3e0fd1bb8d1e"
server_key_iv: "cb91453dccf3f83d03bab6a32c2fa4e1"
server_key_version: "encryptionkey@0"
I20260808 22:55:26.888332 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "881456130cff477888b200c593107b56"
format_stamp: "Formatted at 2026-08-08 22:55:26 on dist-test-slave-1x4g"
server_key: "9f087b475e2db6928b1e3e0fd1bb8d1e"
server_key_iv: "cb91453dccf3f83d03bab6a32c2fa4e1"
server_key_version: "encryptionkey@0"
I20260808 22:55:26.890081 10787 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 22:55:26.890919 10802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:26.891119 10787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:26.891240 10787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "881456130cff477888b200c593107b56"
format_stamp: "Formatted at 2026-08-08 22:55:26 on dist-test-slave-1x4g"
server_key: "9f087b475e2db6928b1e3e0fd1bb8d1e"
server_key_iv: "cb91453dccf3f83d03bab6a32c2fa4e1"
server_key_version: "encryptionkey@0"
I20260808 22:55:26.891330 10787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:26.930099 10787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:26.932415 10787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:26.932600 10787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:26.937253 10787 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37125
I20260808 22:55:26.937299 10861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37125 every 8 connection(s)
I20260808 22:55:26.937713 10787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:26.938663 10862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:26.941241 10862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Bootstrap starting.
I20260808 22:55:26.941875 10862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:26.942207 10862 log.cc:824] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:26.942981 10862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: No bootstrap required, opened a new log
I20260808 22:55:26.943296 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10787
I20260808 22:55:26.943392 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:26.943603 1872 external_mini_cluster.cc:1428] Setting key b522516d74079cb8a1341425fb91a734
I20260808 22:55:26.945529 10862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:26.945745 10862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:26.945787 10862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 881456130cff477888b200c593107b56, State: Initialized, Role: FOLLOWER
I20260808 22:55:26.945940 10862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:26.946035 10862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:26.946075 10862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:26.946138 10862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:26.946753 10862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:26.946851 10862 leader_election.cc:302] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 881456130cff477888b200c593107b56; no voters:
I20260808 22:55:26.947108 10862 leader_election.cc:288] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:26.947130 10867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:26.947260 10867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 LEADER]: Becoming Leader. State: Replica: 881456130cff477888b200c593107b56, State: Running, Role: LEADER
I20260808 22:55:26.947362 10867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:26.947423 10862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:26.947713 10867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "881456130cff477888b200c593107b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } } }
I20260808 22:55:26.947885 10867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:26.948119 10867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 881456130cff477888b200c593107b56. Latest consensus state: current_term: 1 leader_uuid: "881456130cff477888b200c593107b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } } }
I20260808 22:55:26.948205 10867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: This master's current role is: LEADER
W20260808 22:55:26.949579 10884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:26.949707 10884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:26.949820 10874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:26.950107 10874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:55:26 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:26.952270 10874 catalog_manager.cc:1347] Generated new cluster ID: 5bc2d67ecc4e4d4c987233c47b3f4e55
I20260808 22:55:26.952323 10874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:26.957216 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:26.944161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.1.212.62:37125)
0808 22:55:26.944359 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:26.944367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:26.944393 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:55:26.945038 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:26.945046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:26.945099 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:26.945219 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:26.946081 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:26.947623 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:26.948924 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:26.949526 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:26.949653 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:26.952826 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:26.952831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:26.952831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:26.952846 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:26.955206 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:26.955674 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:26.955679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:26.955679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:26.955812 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:26.956035 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:26.956036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:26.956037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:26.956406 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:26.956608 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:26.956770 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 22:55:26.956983 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260808 22:55:26.958516 10874 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:26.959050 10874 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:26.963857 10874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Generated new TSK 0
I20260808 22:55:26.964067 10874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:26.996985 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:27.087026 10889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:27.087284 10889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:27.087313 10889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:27.088654 10889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:27.088716 10889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:27.088747 10889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:27.090484 10889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:27.090813 10889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:27.091087 10889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:27.096642 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.096655 10897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.096657 10894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.096899 10889 server_base.cc:1034] running on GCE node
I20260808 22:55:27.097157 10889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:27.097426 10889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:27.098608 10889 hybrid_clock.cc:648] HybridClock initialized: now 1786229727098588 us; error 42 us; skew 500 ppm
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:27.101528 10889 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:27.102077 10889 webserver.cc:460] Webserver started at http://127.1.212.1:41187/ using document root <none> and password file <none>
I20260808 22:55:27.102293 10889 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:27.102335 10889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:27.102433 10889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:27.103507 10889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "d9661af64d8f41579126b36527c24fb1"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "d47bd62537f308850f30c9be8818a3c1"
server_key_iv: "a361d72bd8022f5ecc90d47211cd8d18"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.103832 10889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "d9661af64d8f41579126b36527c24fb1"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "d47bd62537f308850f30c9be8818a3c1"
server_key_iv: "a361d72bd8022f5ecc90d47211cd8d18"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.105286 10889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:27.106056 10905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.106294 10889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:27.106379 10889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "d9661af64d8f41579126b36527c24fb1"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "d47bd62537f308850f30c9be8818a3c1"
server_key_iv: "a361d72bd8022f5ecc90d47211cd8d18"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.106441 10889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:27.115147 10889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:27.117653 10889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:27.117779 10889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:27.117981 10889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:27.118312 10889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:27.118350 10889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.118374 10889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:27.118419 10889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.124709 10889 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:43861
I20260808 22:55:27.124760 11030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:43861 every 8 connection(s)
I20260808 22:55:27.125123 10889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:27.133682 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 10889
I20260808 22:55:27.133828 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:27.134028 1872 external_mini_cluster.cc:1428] Setting key fe51fc0f1dd922af251ae394a23289eb
I20260808 22:55:27.135375 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.125959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:39389 (local address 127.1.212.62:37125)
0808 22:55:27.126010 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.126012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.126257 (+ 245us) server_negotiation.cc:407] Connection header received
0808 22:55:27.126841 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.126842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.126855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.126917 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.128011 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.128533 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.129653 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.129880 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.129964 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.132340 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:27.132341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:27.132342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:27.132351 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:27.133723 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.134198 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.134199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.134200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.134264 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.134564 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.134566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.134566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.134744 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:27.134829 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:27.135206 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 22:55:27.135306 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:55:27.135687 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.125926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:39389)
0808 22:55:27.126123 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.126163 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.126721 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.126992 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.126996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.127665 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.127890 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.127896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.128658 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.128659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.129544 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.129546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.129978 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.129979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.130125 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.130652 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.130662 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.132232 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:27.133818 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.133828 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.133830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.134129 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.134359 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.134360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.134360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.134490 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.134829 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:27.134836 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:27.135044 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 22:55:27.135184 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 22:55:27.135381 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260808 22:55:27.136276 11031 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37125
I20260808 22:55:27.136382 11031 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:27.136610 11031 heartbeater.cc:511] Master 127.1.212.62:37125 requested a full tablet report, sending...
I20260808 22:55:27.137249 10823 ts_manager.cc:194] Registered new tserver with Master: d9661af64d8f41579126b36527c24fb1 (127.1.212.1:43861)
I20260808 22:55:27.137892 10823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:39389
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:27.163957 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:27.252995 11036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:27.253257 11036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:27.253297 11036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:27.254560 11036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:27.254645 11036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:27.254693 11036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:27.256319 11036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:27.256630 11036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:27.256932 11036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:27.262447 11044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.262481 11041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.262441 11042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.262485 11036 server_base.cc:1034] running on GCE node
I20260808 22:55:27.263043 11036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:27.263346 11036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:27.264523 11036 hybrid_clock.cc:648] HybridClock initialized: now 1786229727264507 us; error 47 us; skew 500 ppm
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:27.267345 11036 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:27.267784 11036 webserver.cc:460] Webserver started at http://127.1.212.2:39005/ using document root <none> and password file <none>
I20260808 22:55:27.267998 11036 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:27.268042 11036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:27.268143 11036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:27.269309 11036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "def100720f5c4cbdbdc4f02ad2bb5d48"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "40384438f55628ef3d45ab77a0ea9224"
server_key_iv: "fb39093a8e2ab40a2bdce17e43e29150"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.269655 11036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "def100720f5c4cbdbdc4f02ad2bb5d48"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "40384438f55628ef3d45ab77a0ea9224"
server_key_iv: "fb39093a8e2ab40a2bdce17e43e29150"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.271302 11036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:55:27.272135 11051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.272348 11036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:27.272477 11036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "def100720f5c4cbdbdc4f02ad2bb5d48"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "40384438f55628ef3d45ab77a0ea9224"
server_key_iv: "fb39093a8e2ab40a2bdce17e43e29150"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.272572 11036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:27.281721 11036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:27.283810 11036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:27.283967 11036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:27.284251 11036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:27.284601 11036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:27.284662 11036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.284704 11036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:27.284732 11036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.291055 11036 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44715
I20260808 22:55:27.291145 11172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44715 every 8 connection(s)
I20260808 22:55:27.291499 11036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:27.300568 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11036
I20260808 22:55:27.300714 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:27.300951 1872 external_mini_cluster.cc:1428] Setting key 6a126e12df7c02c5176f815d8ac0b80e
I20260808 22:55:27.301797 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.292338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:43291 (local address 127.1.212.62:37125)
0808 22:55:27.292388 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.292390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.292626 (+ 236us) server_negotiation.cc:407] Connection header received
0808 22:55:27.293200 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.293203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.293216 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.293256 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.294435 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.295095 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.296143 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.296356 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.296437 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.298716 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:27.298718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:27.298718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:27.298726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:27.300040 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.300535 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.300537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.300537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.300610 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.300972 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.300974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.300974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.301147 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:27.301230 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:27.301589 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 22:55:27.301690 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:55:27.302017 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.292307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:43291)
0808 22:55:27.292496 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.292534 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.293088 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.293344 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.293348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.294095 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.294307 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.294313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.295207 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.295210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.296031 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.296034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.296449 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.296451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.296582 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.297134 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.297143 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.298606 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:27.300164 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.300174 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.300176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.300450 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.300690 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.300691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.300692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.300883 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.301247 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:27.301256 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:27.301424 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 22:55:27.301517 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:55:27.301737 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260808 22:55:27.302516 11173 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37125
I20260808 22:55:27.302639 11173 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:27.302837 11173 heartbeater.cc:511] Master 127.1.212.62:37125 requested a full tablet report, sending...
I20260808 22:55:27.303277 10823 ts_manager.cc:194] Registered new tserver with Master: def100720f5c4cbdbdc4f02ad2bb5d48 (127.1.212.2:44715)
I20260808 22:55:27.303814 10823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:43291
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:27.332028 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:27.421842 11178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:27.422114 11178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:27.422145 11178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:27.423455 11178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:27.423514 11178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:27.423547 11178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:27.425320 11178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:27.425590 11178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:27.425864 11178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:27.431442 11184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.431517 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.431435 11183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.431551 11178 server_base.cc:1034] running on GCE node
I20260808 22:55:27.431849 11178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:27.432085 11178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:27.433229 11178 hybrid_clock.cc:648] HybridClock initialized: now 1786229727433213 us; error 37 us; skew 500 ppm
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:27.436137 11178 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:27.436755 11178 webserver.cc:460] Webserver started at http://127.1.212.3:33861/ using document root <none> and password file <none>
I20260808 22:55:27.437081 11178 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:27.437131 11178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:27.437306 11178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:27.438460 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "baeada96da5c4352938feb56324c9420"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "03327ae907134f2b0933fd0ae881a84b"
server_key_iv: "97fbada54ac576bf0983e331ad44eb55"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.438859 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "baeada96da5c4352938feb56324c9420"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "03327ae907134f2b0933fd0ae881a84b"
server_key_iv: "97fbada54ac576bf0983e331ad44eb55"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.440313 11178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:27.441205 11193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.441416 11178 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:27.441521 11178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "baeada96da5c4352938feb56324c9420"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "03327ae907134f2b0933fd0ae881a84b"
server_key_iv: "97fbada54ac576bf0983e331ad44eb55"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.441606 11178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:27.481273 11178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:27.483590 11178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:27.483771 11178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:27.484076 11178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:27.484447 11178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:27.484505 11178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.484541 11178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:27.484570 11178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.491439 11178 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:33429
I20260808 22:55:27.491488 11315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:33429 every 8 connection(s)
I20260808 22:55:27.491863 11178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:27.500731 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11178
I20260808 22:55:27.500870 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:27.501047 1872 external_mini_cluster.cc:1428] Setting key 291850c32d3965012319d720c2ab8261
I20260808 22:55:27.502439 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.492697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:50403 (local address 127.1.212.62:37125)
0808 22:55:27.492816 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.492819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.493039 (+ 220us) server_negotiation.cc:407] Connection header received
0808 22:55:27.493554 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.493555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.493567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.493636 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.494752 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.495293 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.496369 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.496572 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.496645 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.499152 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:27.499154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:27.499154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:27.499164 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:27.500468 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.501092 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.501093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.501094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.501192 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:27.501550 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:27.501552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:27.501553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:27.501764 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:27.501843 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:27.502231 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:55:27.502336 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:55:27.502643 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.492660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:50403)
0808 22:55:27.492892 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.492941 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.493459 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.493725 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.493730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.494410 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.494660 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.494666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.495376 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.495377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.496279 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.496282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.496670 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.496670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.496820 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.497439 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.497449 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.499025 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:27.500581 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.500592 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.500595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.501008 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.501277 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:27.501278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:27.501278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:27.501439 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:27.501848 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:27.501855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:27.502055 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 22:55:27.502190 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 22:55:27.502379 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":144,"threads_started":1}
I20260808 22:55:27.503105 11316 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37125
I20260808 22:55:27.503239 11316 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:27.503443 11316 heartbeater.cc:511] Master 127.1.212.62:37125 requested a full tablet report, sending...
I20260808 22:55:27.503806 10823 ts_manager.cc:194] Registered new tserver with Master: baeada96da5c4352938feb56324c9420 (127.1.212.3:33429)
I20260808 22:55:27.504160 10823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:50403
I20260808 22:55:27.513412 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
Entry for principal oryx/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab.
WARNING: no policy specified for oryx/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
Entry for principal oryx/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab.
WARNING: no policy specified for oryx/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
Entry for principal oryx/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab.
I20260808 22:55:27.600116 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10889
I20260808 22:55:27.609724 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11036
I20260808 22:55:27.616146 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11178
I20260808 22:55:27.623011 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 10787
I20260808 22:55:27.631276 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37125
--webserver_interface=127.1.212.62
--webserver_port=35199
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:27.717751 11326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:27.718036 11326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:27.718080 11326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:27.719384 11326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:27.719480 11326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:27.719507 11326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:27.719537 11326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:27.719563 11326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:27.721251 11326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=35199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:27.721583 11326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:27.721845 11326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:27.727567 11331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.727569 11334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.727942 11326 server_base.cc:1034] running on GCE node
W20260808 22:55:27.727569 11332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.728273 11326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:27.728662 11326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:27.729851 11326 hybrid_clock.cc:648] HybridClock initialized: now 1786229727729779 us; error 87 us; skew 500 ppm
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:27.732729 11326 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:27.733269 11326 webserver.cc:460] Webserver started at http://127.1.212.62:35199/ using document root <none> and password file <none>
I20260808 22:55:27.733604 11326 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:27.733701 11326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:27.735167 11326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:27.735913 11342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.736106 11326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:27.736227 11326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "881456130cff477888b200c593107b56"
format_stamp: "Formatted at 2026-08-08 22:55:26 on dist-test-slave-1x4g"
server_key: "9f087b475e2db6928b1e3e0fd1bb8d1e"
server_key_iv: "cb91453dccf3f83d03bab6a32c2fa4e1"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.736626 11326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:27.752349 11326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:27.754638 11326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:27.754766 11326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:27.759196 11326 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37125
I20260808 22:55:27.759259 11398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37125 every 8 connection(s)
I20260808 22:55:27.759681 11326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:27.761387 11399 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:55:27.762143 11399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Bootstrap starting.
I20260808 22:55:27.767351 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11326
I20260808 22:55:27.767702 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:43861
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:37125
--webserver_port=41187
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:55:27.771514 11399 log.cc:824] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:27.773522 11399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:55:27.773839 11399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Bootstrap complete.
I20260808 22:55:27.776268 11399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:27.776546 11399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 881456130cff477888b200c593107b56, State: Initialized, Role: FOLLOWER
I20260808 22:55:27.776816 11399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:27.777072 11399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:27.777141 11399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:27.777195 11399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:55:27.778512 11399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:27.778644 11399 leader_election.cc:302] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 881456130cff477888b200c593107b56; no voters:
I20260808 22:55:27.778877 11399 leader_election.cc:288] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:55:27.778921 11404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:55:27.779085 11404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [term 2 LEADER]: Becoming Leader. State: Replica: 881456130cff477888b200c593107b56, State: Running, Role: LEADER
I20260808 22:55:27.779212 11399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:27.779251 11404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } }
I20260808 22:55:27.779666 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "881456130cff477888b200c593107b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } } }
I20260808 22:55:27.779763 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:27.779996 11406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 881456130cff477888b200c593107b56. Latest consensus state: current_term: 2 leader_uuid: "881456130cff477888b200c593107b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "881456130cff477888b200c593107b56" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37125 } } }
I20260808 22:55:27.780115 11406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56 [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:27.780233 11410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:27.780853 11410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:27.781085 11410 catalog_manager.cc:1259] Loaded cluster ID: 5bc2d67ecc4e4d4c987233c47b3f4e55
I20260808 22:55:27.781248 11410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:27.782089 11410 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:27.782348 11410 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 881456130cff477888b200c593107b56: Loaded TSK: 0
I20260808 22:55:27.782593 11410 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:55:27.874111 11401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:27.874394 11401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:27.874449 11401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:27.875852 11401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:27.875936 11401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:27.875984 11401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:27.877837 11401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.1.keytab
--principal=oryx/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:43861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=41187
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:27.878170 11401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:27.878433 11401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:27.884493 11428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.884701 11431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:27.884521 11429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:27.884838 11401 server_base.cc:1034] running on GCE node
I20260808 22:55:27.885100 11401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:27.885428 11401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:27.886649 11401 hybrid_clock.cc:648] HybridClock initialized: now 1786229727886621 us; error 40 us; skew 500 ppm
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229727, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:27.890151 11401 init.cc:377] Logged in from keytab as oryx/127.1.212.1@KRBTEST.COM (short username oryx)
I20260808 22:55:27.890710 11401 webserver.cc:460] Webserver started at http://127.1.212.1:41187/ using document root <none> and password file <none>
I20260808 22:55:27.890986 11401 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:27.891081 11401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:27.892653 11401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:27.893436 11438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.893651 11401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:55:27.893764 11401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "d9661af64d8f41579126b36527c24fb1"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "d47bd62537f308850f30c9be8818a3c1"
server_key_iv: "a361d72bd8022f5ecc90d47211cd8d18"
server_key_version: "encryptionkey@0"
I20260808 22:55:27.894208 11401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:27.913303 11401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:27.915854 11401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:27.916060 11401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:27.916410 11401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:27.916858 11401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:27.916922 11401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.916968 11401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:27.916996 11401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:27.923605 11401 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:43861
I20260808 22:55:27.924804 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11401
I20260808 22:55:27.923673 11562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:43861 every 8 connection(s)
I20260808 22:55:27.924225 11401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:27.925187 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:44715
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:37125
--webserver_port=39005
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:27.937695 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.926047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:50695)
0808 22:55:27.926252 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.926306 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.926960 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.927998 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.928003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.928666 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.928965 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.928972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.930758 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.930760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.932261 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.932266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.933161 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.933163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.933355 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.934082 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.934094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.937417 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260808 22:55:27.937772 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:27.927192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:50695 (local address 127.1.212.62:37125)
0808 22:55:27.927471 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.927485 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.927533 (+ 48us) server_negotiation.cc:407] Connection header received
0808 22:55:27.927681 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.927688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.927745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.927894 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.929109 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.930563 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.932386 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.933044 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.933168 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.937598 (+ 4430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:50695: BlockingRecv error: recv got EOF from 127.1.212.1:50695 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":125,"threads_started":1}
W20260808 22:55:27.938446 11563 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:37125 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.212.62:37125: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:27.944253 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.938748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:53633)
0808 22:55:27.938805 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.938818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.938864 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.939046 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.939047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.939232 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.939305 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.939307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.940206 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.940208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.941148 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.941151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.941511 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.941512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.941594 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.942044 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.942050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.944155 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:55:27.944334 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:27.938804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:53633 (local address 127.1.212.62:37125)
0808 22:55:27.938849 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.938851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.938861 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:27.938939 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.938939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.938953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.939020 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.939410 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.940137 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.941243 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.941436 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.941490 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.944266 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:53633: BlockingRecv error: recv got EOF from 127.1.212.1:53633 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:27 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:27.949803 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:27.945040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:43699)
0808 22:55:27.945172 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:27.945186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:27.945249 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:27.945416 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:27.945417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:27.945568 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:27.945615 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.945616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.946286 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.946287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.946811 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:27.946813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.947193 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.947194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:27.947255 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.947733 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:27.947745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:27.949726 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:55:27.949934 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:27.945121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:43699 (local address 127.1.212.62:37125)
0808 22:55:27.945176 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:27.945178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:27.945241 (+ 63us) server_negotiation.cc:407] Connection header received
0808 22:55:27.945318 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:27.945319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:27.945331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:27.945397 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:27.945698 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.946194 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.946906 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:27.947110 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:27.947191 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:27.949835 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:43699: BlockingRecv error: recv got EOF from 127.1.212.1:43699 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 22:55:27.949967 11563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:55:28.018971 11566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:28.019227 11566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:28.019253 11566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:28.020490 11566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:28.020536 11566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:28.020565 11566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:28.022107 11566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.2.keytab
--principal=oryx/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:44715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=39005
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:28.022336 11566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:28.022550 11566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:28.028357 11572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:28.028388 11573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:28.028388 11575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:28.028827 11566 server_base.cc:1034] running on GCE node
I20260808 22:55:28.029101 11566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:28.029408 11566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:28.030622 11566 hybrid_clock.cc:648] HybridClock initialized: now 1786229728030590 us; error 52 us; skew 500 ppm
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229728, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:28.033752 11566 init.cc:377] Logged in from keytab as oryx/127.1.212.2@KRBTEST.COM (short username oryx)
I20260808 22:55:28.034358 11566 webserver.cc:460] Webserver started at http://127.1.212.2:39005/ using document root <none> and password file <none>
I20260808 22:55:28.034622 11566 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:28.034688 11566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:28.037292 11566 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:55:28.038604 11583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.038825 11566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:55:28.038950 11566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "def100720f5c4cbdbdc4f02ad2bb5d48"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "40384438f55628ef3d45ab77a0ea9224"
server_key_iv: "fb39093a8e2ab40a2bdce17e43e29150"
server_key_version: "encryptionkey@0"
I20260808 22:55:28.039403 11566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:28.064328 11566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:28.067417 11566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:28.067643 11566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:28.067966 11566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:28.068349 11566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:28.068410 11566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.068463 11566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:28.068513 11566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.075538 11566 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44715
I20260808 22:55:28.075595 11709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44715 every 8 connection(s)
I20260808 22:55:28.075981 11566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.083242 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11566
I20260808 22:55:28.083554 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:33429
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:37125
--webserver_port=33861
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:55:28.084652 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.076841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:55943 (local address 127.1.212.62:37125)
0808 22:55:28.076920 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.076922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.077226 (+ 304us) server_negotiation.cc:407] Connection header received
0808 22:55:28.077736 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.077738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.077753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.077810 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.078929 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.079649 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.081039 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.081264 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.081325 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.084593 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:55943: BlockingRecv error: recv got EOF from 127.1.212.2:55943 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:55:28.084717 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.076848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:55943)
0808 22:55:28.077063 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.077116 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.077640 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.077869 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.077873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.078593 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.078814 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.078821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.079753 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.079755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.080939 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.080943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.081359 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.081360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.081499 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.082019 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.082029 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.084487 (+ 2458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
W20260808 22:55:28.084978 11710 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:37125 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.212.62:37125: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.091037 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.085277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:48595)
0808 22:55:28.085332 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.085346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.085392 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.085637 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.085638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.085766 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.085819 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.085821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.086778 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.086779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.087286 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.087287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.087788 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.087789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.087844 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.088392 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.088397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.090899 (+ 2502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:55:28.091180 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.085450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:48595 (local address 127.1.212.62:37125)
0808 22:55:28.085501 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.085504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.085512 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:55:28.085545 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.085546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.085557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.085611 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.085906 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.086665 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.087380 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.087703 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.087780 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.091076 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:48595: BlockingRecv error: recv got EOF from 127.1.212.2:48595 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.097033 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.091348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:37769)
0808 22:55:28.091510 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.091523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.091595 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.091735 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.091736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.091909 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.091965 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.091967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.092691 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.092693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.093639 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.093644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.094090 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.094091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.094163 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.094740 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.094747 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.096970 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
W20260808 22:55:28.097182 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.091466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:37769 (local address 127.1.212.62:37125)
0808 22:55:28.091515 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.091517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.091582 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:55:28.091663 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.091664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.091674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.091726 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.092074 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.092586 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.093788 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.093997 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.094065 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.097109 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:37769: BlockingRecv error: recv got EOF from 127.1.212.2:37769 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 22:55:28.097398 11710 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:55:28.174050 11713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:28.174315 11713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:28.174345 11713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:28.175722 11713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:28.175787 11713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:28.175819 11713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:28.177668 11713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/krb5kdc/oryx_127.1.212.3.keytab
--principal=oryx/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:33429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=33861
--tserver_master_addrs=127.1.212.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:28.177999 11713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:28.178290 11713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:28.184509 11723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:28.184521 11720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:28.184662 11713 server_base.cc:1034] running on GCE node
W20260808 22:55:28.184507 11721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:28.185168 11713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:28.185472 11713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:28.186662 11713 hybrid_clock.cc:648] HybridClock initialized: now 1786229728186651 us; error 37 us; skew 500 ppm
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229728, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:28.189554 11713 init.cc:377] Logged in from keytab as oryx/127.1.212.3@KRBTEST.COM (short username oryx)
I20260808 22:55:28.190076 11713 webserver.cc:460] Webserver started at http://127.1.212.3:33861/ using document root <none> and password file <none>
I20260808 22:55:28.190330 11713 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:28.190429 11713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:28.192287 11713 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:28.193397 11732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.193619 11713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:28.193753 11713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "baeada96da5c4352938feb56324c9420"
format_stamp: "Formatted at 2026-08-08 22:55:27 on dist-test-slave-1x4g"
server_key: "03327ae907134f2b0933fd0ae881a84b"
server_key_iv: "97fbada54ac576bf0983e331ad44eb55"
server_key_version: "encryptionkey@0"
I20260808 22:55:28.194217 11713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:28.208098 11713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:28.211026 11713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:28.211246 11713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:28.211552 11713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:28.211913 11713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:28.211970 11713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.212006 11713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:28.212036 11713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:28.218955 11713 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:33429
I20260808 22:55:28.219005 11859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:33429 every 8 connection(s)
I20260808 22:55:28.219399 11713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:28.221002 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11713
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:55:28.228636 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.220366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34599 (local address 127.1.212.62:37125)
0808 22:55:28.220413 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.220416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.220531 (+ 115us) server_negotiation.cc:407] Connection header received
0808 22:55:28.221133 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.221135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.221149 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.221202 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.222620 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.223169 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.224292 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.224596 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.224700 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.228570 (+ 3870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:34599: BlockingRecv error: recv got EOF from 127.1.212.3:34599 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":21}
I20260808 22:55:28.228736 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.220185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:34599)
0808 22:55:28.220373 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.220429 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.221014 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.221297 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.221301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.222156 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.222497 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.222506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.223283 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.223284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.224154 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.224157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.224716 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.224717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.224887 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.225596 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.225608 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.228495 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
W20260808 22:55:28.228986 11860 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:37125 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.212.62:37125: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
I20260808 22:55:28.231137 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.221605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.1.212.62:37125)
0808 22:55:28.221717 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.221719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.221761 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:55:28.221784 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.221784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.221795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.221843 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.222645 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.223175 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.225051 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.225380 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.225471 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.226990 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:28.226995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:28.226996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:28.227011 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:28.229170 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:28.229645 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:28.229648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:28.229649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:28.229798 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:28.230140 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:28.230142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:28.230142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:28.230494 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:28.230652 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:28.230883 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 22:55:28.230993 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.234503 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.229267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:32977)
0808 22:55:28.229344 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.229354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.229398 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.229526 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.229526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.229641 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.229694 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.229695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.230711 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.230712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.231562 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.231568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.232024 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.232025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.232114 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.232537 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.232542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.234447 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 22:55:28.234908 11567 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.229290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:32977 (local address 127.1.212.62:37125)
0808 22:55:28.229345 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.229347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.229385 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:55:28.229448 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.229449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.229461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.229523 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.229764 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.230612 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.231678 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.231941 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.232024 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.234819 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:32977: BlockingRecv error: recv got EOF from 127.1.212.3:32977 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.239425 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.234853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:36473)
0808 22:55:28.234907 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.234920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.234967 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.235113 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.235114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.235242 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.235279 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.235281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.236046 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.236047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.236557 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.236558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.237028 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.237029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.237092 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.237520 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.237525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.239368 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:55:28.239593 11860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:55:28.239625 11864 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.234943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:36473 (local address 127.1.212.62:37125)
0808 22:55:28.234990 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.234992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.235012 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:55:28.235038 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.235039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.235051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.235097 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.235349 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.235848 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.236682 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.236943 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.237013 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.239558 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:36473: BlockingRecv error: recv got EOF from 127.1.212.3:36473 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:28 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:28.955941 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:28.950496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:37667)
0808 22:55:28.950660 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:28.950671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:28.950728 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:28.950905 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:28.950906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:28.951083 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:28.951151 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.951152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.952099 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.952100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.952670 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:28.952673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.953115 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.953116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:28.953171 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.953674 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:28.953680 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:28.955858 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
W20260808 22:55:28.956110 11866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:28.950585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:37667 (local address 127.1.212.62:37125)
0808 22:55:28.950748 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:28.950750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:28.950761 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:28.950791 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:28.950791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:28.950802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:28.950875 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:28.951225 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.952003 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.952751 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:28.953016 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:28.953092 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:28.955973 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:37667: BlockingRecv error: recv got EOF from 127.1.212.1:37667 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:29.102991 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:29.097924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:51715)
0808 22:55:29.098115 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:29.098127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:29.098186 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:29.098370 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:29.098371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:29.098565 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:29.098618 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.098619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.099272 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.099273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.099833 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.099835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.100183 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.100185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.100228 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.100831 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:29.100837 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:29.102897 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":82,"threads_started":1}
W20260808 22:55:29.103140 11866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:29.098027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51715 (local address 127.1.212.62:37125)
0808 22:55:29.098243 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:29.098245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:29.098254 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:29.098280 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:29.098281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:29.098295 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:29.098346 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:29.098689 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.099198 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.099919 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.100114 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.100170 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.103040 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:51715: BlockingRecv error: recv got EOF from 127.1.212.2:51715 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:29.245000 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:29.240053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:51699)
0808 22:55:29.240177 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:29.240191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:29.240292 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:29.240479 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:29.240480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:29.240745 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:29.240835 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.240837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.241509 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.241510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.242087 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.242088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.242430 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.242431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.242476 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.242949 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:29.242954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:29.244921 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
W20260808 22:55:29.245124 11866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:29.240101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:51699 (local address 127.1.212.62:37125)
0808 22:55:29.240176 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:29.240179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:29.240290 (+ 111us) server_negotiation.cc:407] Connection header received
0808 22:55:29.240368 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:29.240369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:29.240383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:29.240448 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:29.240924 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.241426 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.242169 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.242357 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.242429 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.245024 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:51699: BlockingRecv error: recv got EOF from 127.1.212.3:51699 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:29 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:29.961704 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:29.956550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:44891)
0808 22:55:29.956714 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:29.956729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:29.956824 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:29.957030 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:29.957031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:29.957203 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:29.957258 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.957259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.958030 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.958032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.958603 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:29.958604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.958993 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.958995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:29.959040 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.959567 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:29.959572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:29.961635 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
W20260808 22:55:29.961854 11870 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:29.956636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:44891 (local address 127.1.212.62:37125)
0808 22:55:29.956794 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:29.956796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:29.956808 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:55:29.956911 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:29.956912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:29.956924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:29.956994 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:29.957372 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.957906 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.958698 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:29.958912 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:29.958981 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:29.961757 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:44891: BlockingRecv error: recv got EOF from 127.1.212.1:44891 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:30.108907 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:30.103692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:60287)
0808 22:55:30.103927 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:30.103939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:30.104006 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:30.104168 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:30.104169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:30.104349 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:30.104405 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.104406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.105326 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.105327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.105869 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.105871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.106220 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.106221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.106265 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.106799 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:30.106804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:30.108816 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":155,"threads_started":1}
W20260808 22:55:30.109014 11870 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:30.103890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:60287 (local address 127.1.212.62:37125)
0808 22:55:30.103949 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:30.103951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:30.103990 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:55:30.104084 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:30.104085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:30.104098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:30.104144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:30.104501 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.105220 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.105964 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.106155 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.106204 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.108963 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:60287: BlockingRecv error: recv got EOF from 127.1.212.2:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:30.250348 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:30.245545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:44339)
0808 22:55:30.245740 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:30.245758 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:30.245838 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:30.246000 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:30.246001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:30.246179 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:30.246231 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.246233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.246898 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.246899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.247435 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.247437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.247785 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.247786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.247830 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.248286 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:30.248291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:30.250279 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
W20260808 22:55:30.250507 11870 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:30.245728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:44339 (local address 127.1.212.62:37125)
0808 22:55:30.245798 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:30.245800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:30.245815 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:55:30.245904 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:30.245905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:30.245919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:30.245972 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:30.246306 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.246823 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.247522 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.247712 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.247770 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.250420 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:44339: BlockingRecv error: recv got EOF from 127.1.212.3:44339 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:30 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:30.968657 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:30.962496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:50859)
0808 22:55:30.962776 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:30.962795 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:30.962896 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:30.963123 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:30.963125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:30.963436 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:30.963526 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.963532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.964493 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.964495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.965430 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:30.965435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.965977 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.965979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:30.966047 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.966589 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:30.966594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:30.968563 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":205,"threads_started":1}
W20260808 22:55:30.968747 11874 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:30.962681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:50859 (local address 127.1.212.62:37125)
0808 22:55:30.962935 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:30.962937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:30.962972 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:55:30.963003 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:30.963004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:30.963017 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:30.963097 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:30.963665 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.964390 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.965576 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:30.965871 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:30.965975 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:30.968673 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:50859: BlockingRecv error: recv got EOF from 127.1.212.1:50859 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:31.114665 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:31.109509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:51351)
0808 22:55:31.109777 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:31.109789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:31.109864 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:31.110046 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:31.110046 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:31.110233 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:31.110289 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.110291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.110950 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.110951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.111521 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.111524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.111889 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.111890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.111937 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.112498 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:31.112503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:31.114587 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":77,"threads_started":1}
W20260808 22:55:31.114789 11874 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:31.109568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51351 (local address 127.1.212.62:37125)
0808 22:55:31.109777 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:31.109779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:31.109844 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:55:31.109922 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:31.109923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:31.109935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:31.110001 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:31.110366 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.110866 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.111614 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.111810 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.111870 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.114713 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:51351: BlockingRecv error: recv got EOF from 127.1.212.2:51351 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:31.255996 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:31.250993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:55219)
0808 22:55:31.251206 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:31.251219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:31.251288 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:31.251432 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:31.251433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:31.251627 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:31.251679 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.251680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.252328 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.252329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.252919 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.252921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.253324 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.253325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.253393 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.253911 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:31.253916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:31.255879 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":132,"threads_started":1}
W20260808 22:55:31.256038 11874 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:31.251100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:55219 (local address 127.1.212.62:37125)
0808 22:55:31.251186 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:31.251193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:31.251287 (+ 94us) server_negotiation.cc:407] Connection header received
0808 22:55:31.251350 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:31.251351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:31.251364 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:31.251407 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:31.251748 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.252249 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.253023 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.253228 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.253321 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.255971 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:55219: BlockingRecv error: recv got EOF from 127.1.212.3:55219 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:31 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:31.977541 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:31.969347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:42379)
0808 22:55:31.969505 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:31.969516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:31.969623 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:31.969791 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:31.969792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:31.969971 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:31.970027 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.970028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.970720 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.970721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.971266 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:31.971267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.971690 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.971691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:31.971741 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.975369 (+ 3628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:31.975376 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:31.977465 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
W20260808 22:55:31.977828 11879 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:31.969403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:42379 (local address 127.1.212.62:37125)
0808 22:55:31.969522 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:31.969524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:31.969628 (+ 104us) server_negotiation.cc:407] Connection header received
0808 22:55:31.969679 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:31.969679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:31.969694 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:31.969763 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:31.970113 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.970630 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.971387 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:31.971606 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:31.971677 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:31.977746 (+ 6069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:42379: BlockingRecv error: recv got EOF from 127.1.212.1:42379 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:32.120538 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:32.115297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:51017)
0808 22:55:32.115467 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:32.115486 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:32.115555 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:32.115696 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:32.115697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:32.115874 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:32.115933 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.115934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.116624 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.116625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.117228 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.117231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.117676 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.117678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.117736 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.118312 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:32.118318 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:32.120474 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":112,"threads_started":1}
W20260808 22:55:32.120703 11879 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:32.115388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51017 (local address 127.1.212.62:37125)
0808 22:55:32.115466 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:32.115469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:32.115555 (+ 86us) server_negotiation.cc:407] Connection header received
0808 22:55:32.115605 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:32.115605 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:32.115617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:32.115671 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:32.116011 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.116523 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.117350 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.117558 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.117626 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.120611 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:51017: BlockingRecv error: recv got EOF from 127.1.212.2:51017 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:32.261814 11881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:32.256663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:35953)
0808 22:55:32.256854 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:32.256868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:32.256942 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:32.257114 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:32.257116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:32.257315 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:32.257366 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.257367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.258170 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.258171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.258708 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.258710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.259100 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.259101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.259190 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.259683 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:32.259689 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:32.261751 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":129,"threads_started":1}
W20260808 22:55:32.261983 11879 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:32.256820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:35953 (local address 127.1.212.62:37125)
0808 22:55:32.256907 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:32.256909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:32.256923 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:55:32.257007 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:32.257009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:32.257035 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:32.257105 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:32.257468 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.258073 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.258810 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.259017 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.259099 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.261904 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:35953: BlockingRecv error: recv got EOF from 127.1.212.3:35953 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:32 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:32.983640 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:32.978245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:32877)
0808 22:55:32.978439 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:32.978456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:32.978541 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:32.978713 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:32.978714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:32.978885 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:32.978941 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.978943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.980003 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.980004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.980692 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:32.980695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.981109 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.981110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:32.981154 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.981620 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:32.981625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:32.983552 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
W20260808 22:55:32.984058 11883 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:32.978411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:32877 (local address 127.1.212.62:37125)
0808 22:55:32.978561 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:32.978563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:32.978590 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:55:32.978620 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:32.978621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:32.978633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:32.978691 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:32.979042 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.979912 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.980819 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:32.981037 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:32.981097 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:32.984001 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:32877: BlockingRecv error: recv got EOF from 127.1.212.1:32877 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:33.126446 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:33.121174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:34297)
0808 22:55:33.121352 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:33.121365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:33.121441 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:33.121622 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:33.121623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:33.121810 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:33.121867 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.121869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.122568 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.122568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.123117 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.123118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.123506 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.123508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.123574 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.124092 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:33.124100 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:33.126380 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
W20260808 22:55:33.126595 11883 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:33.121364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:34297 (local address 127.1.212.62:37125)
0808 22:55:33.121448 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:33.121450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:33.121495 (+ 45us) server_negotiation.cc:407] Connection header received
0808 22:55:33.121527 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:33.121529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:33.121540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:33.121591 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:33.121952 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.122471 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.123206 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.123401 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.123463 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.126514 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:34297: BlockingRecv error: recv got EOF from 127.1.212.2:34297 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:33.267223 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:33.262364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:40717)
0808 22:55:33.262541 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:33.262552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:33.262622 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:33.262774 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:33.262775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:33.262950 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:33.263002 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.263004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.263713 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.263714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.264243 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.264245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.264606 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.264607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.264650 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.265256 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:33.265261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:33.267161 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
W20260808 22:55:33.267388 11883 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:33.262462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:40717 (local address 127.1.212.62:37125)
0808 22:55:33.262553 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:33.262557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:33.262607 (+ 50us) server_negotiation.cc:407] Connection header received
0808 22:55:33.262689 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:33.262690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:33.262703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:33.262748 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:33.263104 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.263631 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.264332 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.264528 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.264587 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.267305 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:40717: BlockingRecv error: recv got EOF from 127.1.212.3:40717 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:33 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:33.989470 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:33.984395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:38385)
0808 22:55:33.984582 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:33.984602 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:33.984659 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:33.984864 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:33.984865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:33.985043 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:33.985102 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.985104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.985825 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.985826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.986391 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:33.986392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.986797 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.986798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:33.986852 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.987382 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:33.987387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:33.989407 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
W20260808 22:55:33.989630 11889 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:33.984475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:38385 (local address 127.1.212.62:37125)
0808 22:55:33.984636 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:33.984638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:33.984650 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:55:33.984729 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:33.984729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:33.984742 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:33.984828 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:33.985216 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.985734 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.986501 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:33.986702 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:33.986759 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:33.989534 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:38385: BlockingRecv error: recv got EOF from 127.1.212.1:38385 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:34.132575 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:34.127026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:42383)
0808 22:55:34.127233 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:34.127249 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:34.127336 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:34.127511 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:34.127512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:34.127724 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:34.127774 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.127776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.128511 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.128512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.129122 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.129125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.129511 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.129512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.129581 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.130160 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:34.130165 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:34.132470 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":109,"threads_started":1}
W20260808 22:55:34.132622 11889 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:34.127145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:42383 (local address 127.1.212.62:37125)
0808 22:55:34.127238 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:34.127241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:34.127319 (+ 78us) server_negotiation.cc:407] Connection header received
0808 22:55:34.127404 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:34.127405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:34.127420 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:34.127500 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:34.127868 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.128400 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.129222 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.129428 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.129508 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.132555 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:42383: BlockingRecv error: recv got EOF from 127.1.212.2:42383 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:34.272918 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:34.267781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:34107)
0808 22:55:34.267998 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:34.268017 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:34.268103 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:34.268292 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:34.268293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:34.268496 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:34.268549 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.268550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.269318 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.269319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.269856 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.269858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.270239 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.270240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.270300 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.270806 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:34.270811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:34.272789 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":130,"threads_started":1}
W20260808 22:55:34.272977 11889 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:34.267889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34107 (local address 127.1.212.62:37125)
0808 22:55:34.267978 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:34.267980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:34.268082 (+ 102us) server_negotiation.cc:407] Connection header received
0808 22:55:34.268188 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:34.268190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:34.268205 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:34.268282 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:34.268636 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.269221 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.269956 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.270161 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.270241 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.272908 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:34107: BlockingRecv error: recv got EOF from 127.1.212.3:34107 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:34 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:34.995041 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:34.990010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:59645)
0808 22:55:34.990183 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:34.990197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:34.990275 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:34.990444 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:34.990445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:34.990624 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:34.990684 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.990686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.991386 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.991387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.991931 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:34.991933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.992303 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.992304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:34.992357 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.992965 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:34.992971 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:34.994931 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":123,"threads_started":1}
W20260808 22:55:34.995132 11893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:34.990077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:59645 (local address 127.1.212.62:37125)
0808 22:55:34.990225 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:34.990227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:34.990255 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:55:34.990338 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:34.990339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:34.990350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:34.990414 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:34.990760 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.991295 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.992012 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:34.992209 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:34.992262 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:34.995042 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:59645: BlockingRecv error: recv got EOF from 127.1.212.1:59645 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:35.138047 11894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:35.133212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:45603)
0808 22:55:35.133404 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:35.133424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:35.133489 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:35.133638 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:35.133639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:35.133808 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:35.133862 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.133863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.134526 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.134527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.135085 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.135086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.135464 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.135465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.135513 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:35.136067 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:35.136072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:35.137984 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":126,"threads_started":1}
W20260808 22:55:35.138229 11893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:35.133285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:45603 (local address 127.1.212.62:37125)
0808 22:55:35.133337 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:35.133339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:35.133486 (+ 147us) server_negotiation.cc:407] Connection header received
0808 22:55:35.133537 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:35.133538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:35.133551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:35.133611 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:35.133925 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.134438 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.135172 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.135387 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.135445 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:35.138150 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:45603: BlockingRecv error: recv got EOF from 127.1.212.2:45603 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:35.278530 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:35.273531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:43269)
0808 22:55:35.273755 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:35.273766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:35.273838 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:35.273974 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:35.273974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:35.274188 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:35.274240 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.274242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.274945 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.274946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.275489 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.275491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.275877 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.275878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.275942 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:35.276509 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:35.276513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:35.278420 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":136,"threads_started":1}
W20260808 22:55:35.278600 11893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:35.273637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:43269 (local address 127.1.212.62:37125)
0808 22:55:35.273731 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:35.273733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:35.273816 (+ 83us) server_negotiation.cc:407] Connection header received
0808 22:55:35.273894 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:35.273895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:35.273912 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:35.273958 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:35.274329 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.274853 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.275581 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.275783 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.275872 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:35.278517 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:43269: BlockingRecv error: recv got EOF from 127.1.212.3:43269 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:35 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:36.001296 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:35.995755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:49055)
0808 22:55:35.995952 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:35.995963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:35.996027 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:35.996251 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:35.996253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:35.996436 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:35.996506 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.996508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.997469 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.997470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.998013 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:35.998014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.998478 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.998479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:35.998535 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:35.999102 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:35.999108 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:36.001229 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
W20260808 22:55:36.001543 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:35.995925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:49055 (local address 127.1.212.62:37125)
0808 22:55:35.996076 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:35.996078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:35.996089 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:35.996128 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:35.996129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:35.996144 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:35.996215 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:35.996604 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.997361 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.998113 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:35.998335 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:35.998445 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:36.001467 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:49055: BlockingRecv error: recv got EOF from 127.1.212.1:49055 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
Aug 08 22:55:36 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:36 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:36.143615 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:36.138580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:37313)
0808 22:55:36.138731 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:36.138758 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:36.138842 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:36.138993 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:36.138994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:36.139190 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:36.139243 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:36.139245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.140126 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.140127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:36.140684 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:36.140686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.141047 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.141048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:36.141093 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:36.141568 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:36.141573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:36.143551 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
W20260808 22:55:36.143777 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:36.138673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:37313 (local address 127.1.212.62:37125)
0808 22:55:36.138775 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:36.138777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:36.138833 (+ 56us) server_negotiation.cc:407] Connection header received
0808 22:55:36.138890 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:36.138891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:36.138905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:36.138957 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:36.139309 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.140024 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.140760 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.140975 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.141033 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:36.143698 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:37313: BlockingRecv error: recv got EOF from 127.1.212.2:37313 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 22:55:36 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:36 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:36.284651 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:36.279174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:55297)
0808 22:55:36.279555 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:36.279567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:36.279692 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:36.279913 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:36.279914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:36.280144 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:36.280207 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:36.280209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.281183 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.281185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:36.281742 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:36.281744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.282080 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.282080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:36.282127 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:36.282706 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:36.282712 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:36.284588 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":286,"threads_started":1}
W20260808 22:55:36.284888 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:36.279306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:55297 (local address 127.1.212.62:37125)
0808 22:55:36.279376 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:36.279378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:36.279668 (+ 290us) server_negotiation.cc:407] Connection header received
0808 22:55:36.279822 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:36.279823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:36.279838 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:36.279892 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:36.280335 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.281093 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.281819 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:36.282005 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:36.282058 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:36.284776 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:55297: BlockingRecv error: recv got EOF from 127.1.212.3:55297 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:37.012451 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:37.001876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:56355)
0808 22:55:37.002082 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:37.002094 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:37.002182 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:37.002394 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:37.002395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:37.002674 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:37.002737 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.002739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.003540 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.003541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.004084 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.004086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.004428 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.004429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.004470 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.010204 (+ 5734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:37.010212 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:37.012381 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":108,"threads_started":1}
W20260808 22:55:37.012673 11902 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:37.001958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56355 (local address 127.1.212.62:37125)
0808 22:55:37.002079 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:37.002081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:37.002184 (+ 103us) server_negotiation.cc:407] Connection header received
0808 22:55:37.002284 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:37.002285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:37.002303 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:37.002386 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:37.002847 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.003448 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.004160 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.004354 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.004409 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.012596 (+ 8187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:56355: BlockingRecv error: recv got EOF from 127.1.212.1:56355 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:37.149636 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:37.144111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:54459)
0808 22:55:37.144329 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:37.144343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:37.144457 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:37.144634 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:37.144635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:37.144861 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:37.144915 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.144918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.145647 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.145649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.146190 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.146192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.146599 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.146600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.146673 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.147188 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:37.147194 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:37.149549 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
W20260808 22:55:37.149744 11902 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:37.144231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:54459 (local address 127.1.212.62:37125)
0808 22:55:37.144332 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:37.144334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:37.144433 (+ 99us) server_negotiation.cc:407] Connection header received
0808 22:55:37.144527 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:37.144528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:37.144542 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:37.144623 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:37.145010 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.145555 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.146287 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.146501 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.146584 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.149674 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:54459: BlockingRecv error: recv got EOF from 127.1.212.2:54459 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:37 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:37.290171 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:37.285225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:35243)
0808 22:55:37.285457 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:37.285467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:37.285530 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:37.285682 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:37.285682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:37.285867 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:37.285925 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.285926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.286580 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.286581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.287142 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:37.287144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.287502 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.287503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:37.287545 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.288104 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:37.288109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:37.290093 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":138,"threads_started":1}
W20260808 22:55:37.290372 11902 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:37.285336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:35243 (local address 127.1.212.62:37125)
0808 22:55:37.285406 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:37.285408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:37.285529 (+ 121us) server_negotiation.cc:407] Connection header received
0808 22:55:37.285598 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:37.285599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:37.285613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:37.285657 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:37.285990 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.286496 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.287231 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:37.287421 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:37.287485 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:37.290262 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:35243: BlockingRecv error: recv got EOF from 127.1.212.3:35243 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:38.018512 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:38.013116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:48191)
0808 22:55:38.013277 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:38.013297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:38.013364 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:38.013542 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:38.013543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:38.013732 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:38.013790 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.013791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.014640 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.014641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.015189 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.015190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.015549 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.015549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.015591 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.016171 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:38.016177 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:38.018372 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
W20260808 22:55:38.018553 11908 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:38.013245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:48191 (local address 127.1.212.62:37125)
0808 22:55:38.013381 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:38.013388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:38.013398 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:38.013427 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:38.013427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:38.013438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:38.013518 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:38.013898 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.014545 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.015269 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.015476 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.015535 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.018476 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:48191: BlockingRecv error: recv got EOF from 127.1.212.1:48191 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:38.155488 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:38.150308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:55107)
0808 22:55:38.150487 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:38.150500 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:38.150578 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:38.150772 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:38.150773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:38.150984 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:38.151040 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.151042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.151755 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.151757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.152308 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.152310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.152739 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.152740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.152833 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.153372 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:38.153377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:38.155371 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
W20260808 22:55:38.155548 11908 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:38.150400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:55107 (local address 127.1.212.62:37125)
0808 22:55:38.150475 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:38.150478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:38.150553 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:55:38.150654 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:38.150655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:38.150670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:38.150762 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:38.151130 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.151654 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.152431 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.152640 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.152736 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.155474 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:55107: BlockingRecv error: recv got EOF from 127.1.212.2:55107 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:38 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:38.295652 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:38.290726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:43891)
0808 22:55:38.290907 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:38.290921 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:38.291038 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:38.291208 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:38.291209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:38.291408 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:38.291457 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.291459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.292158 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.292159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.292727 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:38.292730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.293174 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.293175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:38.293228 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.293774 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:38.293779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:38.295590 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
W20260808 22:55:38.295800 11908 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:38.290925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:43891 (local address 127.1.212.62:37125)
0808 22:55:38.290994 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:38.290996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:38.291013 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:55:38.291100 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:38.291102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:38.291121 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:38.291197 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:38.291571 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.292066 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.292866 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:38.293087 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:38.293152 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:38.295718 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:43891: BlockingRecv error: recv got EOF from 127.1.212.3:43891 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:39.024257 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:39.019128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:40043)
0808 22:55:39.019303 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:39.019315 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:39.019366 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:39.019597 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:39.019599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:39.019792 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:39.019844 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.019845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.020570 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.020571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.021202 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.021204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.021612 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.021613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.021668 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.022237 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:39.022242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:39.024175 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
W20260808 22:55:39.024446 11912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:39.019247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:40043 (local address 127.1.212.62:37125)
0808 22:55:39.019404 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:39.019406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:39.019417 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:39.019455 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:39.019455 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:39.019469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:39.019547 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:39.019953 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.020486 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.021301 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.021517 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.021583 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.024318 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:40043: BlockingRecv error: recv got EOF from 127.1.212.1:40043 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:39.161603 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:39.156224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:56317)
0808 22:55:39.156419 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:39.156431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:39.156495 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:39.156672 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:39.156673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:39.156880 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:39.156931 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.156932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.157606 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.157607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.158167 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.158169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.158528 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.158529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.158577 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.159093 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:39.159099 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:39.161482 (+ 2383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
W20260808 22:55:39.161690 11912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:39.156379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:56317 (local address 127.1.212.62:37125)
0808 22:55:39.156454 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:39.156456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:39.156471 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:55:39.156555 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:39.156556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:39.156569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:39.156646 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:39.156999 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.157516 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.158251 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.158448 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.158519 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.161608 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:56317: BlockingRecv error: recv got EOF from 127.1.212.2:56317 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:39 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:39.301514 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:39.296164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:56609)
0808 22:55:39.296355 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:39.296368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:39.296480 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:39.296682 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:39.296683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:39.296971 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:39.297062 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.297077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.297828 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.297829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.298572 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:39.298581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.298967 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.298968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:39.299027 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.299500 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:39.299505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:39.301438 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
W20260808 22:55:39.301625 11912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:39.296259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:56609 (local address 127.1.212.62:37125)
0808 22:55:39.296352 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:39.296355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:39.296455 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:55:39.296573 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:39.296574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:39.296591 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:39.296673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:39.297189 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.297734 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.298669 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:39.298875 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:39.298963 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:39.301546 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:56609: BlockingRecv error: recv got EOF from 127.1.212.3:56609 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:40.029510 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:40.024967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:39839)
0808 22:55:40.025101 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:40.025114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:40.025175 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:40.025339 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:40.025340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:40.025517 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:40.025569 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.025570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.026233 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.026234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.026754 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.026756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.027100 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.027101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.027143 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.027624 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:40.027628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:40.029434 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
W20260808 22:55:40.029618 11916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:40.025061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:39839 (local address 127.1.212.62:37125)
0808 22:55:40.025192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:40.025194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:40.025205 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:40.025239 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:40.025240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:40.025252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:40.025316 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:40.025639 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.026160 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.026831 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.027031 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.027100 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.029537 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:39839: BlockingRecv error: recv got EOF from 127.1.212.1:39839 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:40.167315 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:40.162214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:45923)
0808 22:55:40.162448 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:40.162460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:40.162568 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:40.162837 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:40.162838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:40.163068 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:40.163140 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.163143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.163895 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.163896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.164443 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.164445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.164845 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.164846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.164888 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.165349 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:40.165354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:40.167239 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":133,"threads_started":1}
W20260808 22:55:40.167451 11916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:40.162349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:45923 (local address 127.1.212.62:37125)
0808 22:55:40.162427 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:40.162430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:40.162544 (+ 114us) server_negotiation.cc:407] Connection header received
0808 22:55:40.162664 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:40.162665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:40.162679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:40.162761 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:40.163258 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.163813 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.164533 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.164727 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.164851 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.167365 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:45923: BlockingRecv error: recv got EOF from 127.1.212.2:45923 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:40 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:40.307307 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:40.302177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:56201)
0808 22:55:40.302334 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:40.302353 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:40.302407 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:40.302570 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:40.302571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:40.302751 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:40.302802 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.302804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.303543 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.303544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.304127 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:40.304129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.304572 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.304573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:40.304630 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.305229 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:40.305235 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:40.307226 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260808 22:55:40.307435 11916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:40.302305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:56201 (local address 127.1.212.62:37125)
0808 22:55:40.302378 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:40.302380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:40.302394 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:55:40.302474 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:40.302475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:40.302488 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:40.302551 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:40.302910 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.303453 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.304246 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:40.304458 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:40.304530 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:40.307361 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:56201: BlockingRecv error: recv got EOF from 127.1.212.3:56201 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:41.035301 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:41.030165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:53363)
0808 22:55:41.030428 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:41.030453 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:41.030524 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:41.030689 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:41.030690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:41.030865 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:41.030918 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.030920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.031637 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.031639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.032222 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.032224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.032611 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.032612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.032655 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.033170 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:41.033175 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:41.035219 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":197,"threads_started":1}
W20260808 22:55:41.035490 11920 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:41.030405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:53363 (local address 127.1.212.62:37125)
0808 22:55:41.030548 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:41.030550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:41.030560 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:41.030589 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:41.030589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:41.030601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:41.030663 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:41.030996 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.031514 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.032315 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.032530 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.032589 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.035372 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:53363: BlockingRecv error: recv got EOF from 127.1.212.1:53363 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:41.173599 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:41.167948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:60547)
0808 22:55:41.168141 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:41.168155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:41.168253 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:41.168445 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:41.168446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:41.168707 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:41.168761 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.168763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.169518 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.169519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.170064 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.170066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.170480 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.170481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.170543 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.171117 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:41.171131 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:41.173532 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
W20260808 22:55:41.173774 11920 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:41.168102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:60547 (local address 127.1.212.62:37125)
0808 22:55:41.168151 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:41.168154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:41.168234 (+ 80us) server_negotiation.cc:407] Connection header received
0808 22:55:41.168338 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:41.168339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:41.168354 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:41.168433 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:41.168891 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.169416 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.170192 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.170394 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.170479 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.173694 (+ 3215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:60547: BlockingRecv error: recv got EOF from 127.1.212.2:60547 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:41 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:41.313490 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:41.308087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:58393)
0808 22:55:41.308308 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:41.308320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:41.308423 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:41.308624 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:41.308625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:41.308895 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:41.308948 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.308950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.309649 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.309650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.310195 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:41.310197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.310567 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.310568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:41.310640 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.311210 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:41.311215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:41.313404 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":137,"threads_started":1}
W20260808 22:55:41.313606 11920 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:41.308215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:58393 (local address 127.1.212.62:37125)
0808 22:55:41.308297 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:41.308300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:41.308399 (+ 99us) server_negotiation.cc:407] Connection header received
0808 22:55:41.308507 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:41.308509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:41.308524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:41.308602 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:41.309033 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.309556 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.310287 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:41.310489 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:41.310568 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:41.313507 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:58393: BlockingRecv error: recv got EOF from 127.1.212.3:58393 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:42.045574 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:42.035920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:56675)
0808 22:55:42.036108 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:42.036120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:42.036186 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:42.036328 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:42.036329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:42.036507 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:42.036558 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.036560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.037277 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.037278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.037807 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.037809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.038213 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.038214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.038257 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.043065 (+ 4808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:42.043072 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:42.045457 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
W20260808 22:55:42.045719 11924 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:42.035996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56675 (local address 127.1.212.62:37125)
0808 22:55:42.036113 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:42.036115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:42.036189 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:55:42.036236 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:42.036237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:42.036249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:42.036314 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:42.036644 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.037204 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.037909 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.038133 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.038202 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.045582 (+ 7380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:56675: BlockingRecv error: recv got EOF from 127.1.212.1:56675 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:42.179395 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:42.174224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:49925)
0808 22:55:42.174417 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:42.174432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:42.174499 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:42.174663 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:42.174664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:42.174845 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:42.174901 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.174903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.175612 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.175613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.176201 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.176203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.176592 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.176592 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.176652 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.177243 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:42.177248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:42.179223 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":121,"threads_started":1}
W20260808 22:55:42.179430 11924 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:42.174334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:49925 (local address 127.1.212.62:37125)
0808 22:55:42.174393 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:42.174395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:42.174500 (+ 105us) server_negotiation.cc:407] Connection header received
0808 22:55:42.174556 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:42.174557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:42.174570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:42.174634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:42.174974 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.175492 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.176305 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.176507 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.176568 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.179374 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:49925: BlockingRecv error: recv got EOF from 127.1.212.2:49925 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:42 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.3@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:42.318833 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:42.314048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.3:57093)
0808 22:55:42.314227 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:42.314244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:42.314314 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:42.314452 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:42.314452 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:42.314637 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:42.314690 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.314692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.315415 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.315416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.315963 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:42.315966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.316315 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.316316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:42.316375 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.316885 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:42.316890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:42.318769 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
W20260808 22:55:42.319110 11924 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:42.314171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:57093 (local address 127.1.212.62:37125)
0808 22:55:42.314244 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:42.314246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:42.314309 (+ 63us) server_negotiation.cc:407] Connection header received
0808 22:55:42.314361 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:42.314361 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:42.314375 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:42.314426 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:42.314754 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.315338 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.316046 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:42.316241 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:42.316295 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:42.319022 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.3:57093: BlockingRecv error: recv got EOF from 127.1.212.3:57093 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 22:55:43 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:43 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.1@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:43.050979 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.046072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.1:34441)
0808 22:55:43.046240 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:43.046252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:43.046305 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:43.046446 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:43.046446 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:43.046683 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:43.046750 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.046753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.047436 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.047437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.047977 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.047978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.048323 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.048324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.048369 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.048901 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:43.048907 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:43.050859 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
W20260808 22:55:43.051038 11928 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:43.046156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:34441 (local address 127.1.212.62:37125)
0808 22:55:43.046290 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.046292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.046309 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:55:43.046364 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.046364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.046376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.046426 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.046833 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.047363 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.048063 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.048258 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.048308 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.050956 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.1:34441: BlockingRecv error: recv got EOF from 127.1.212.1:34441 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
Aug 08 22:55:43 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:55:43 dist-test-slave-1x4g krb5kdc[10773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.212.2@KRBTEST.COM for oryx/127.1.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:55:43.185230 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.180070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37125 (local address 127.1.212.2:51583)
0808 22:55:43.180256 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:43.180269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:43.180380 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:43.180569 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:43.180570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:43.180763 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:43.180863 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.180865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.181587 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.181588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.182117 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.182118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.182519 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.182520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.182587 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.183083 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:43.183088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:43.185129 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.212.62:37125: Server oryx/127.1.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
W20260808 22:55:43.185298 11928 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:43.180172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:51583 (local address 127.1.212.62:37125)
0808 22:55:43.180255 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.180257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.180357 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:55:43.180454 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.180454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.180485 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.180559 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.180947 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.181471 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.182209 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.182428 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.182516 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.185232 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.212.2:51583: BlockingRecv error: recv got EOF from 127.1.212.2:51583 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:55:43.221850 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11401
I20260808 22:55:43.229149 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11566
I20260808 22:55:43.236531 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11713
I20260808 22:55:43.242794 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11326
2026-08-08T22:55:43Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17686 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T22:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:43Z Disabled control of system clock
I20260808 22:55:43.274828 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:38915
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:55:43.361428 11933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:43.361689 11933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:43.361714 11933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:43.363030 11933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:43.363091 11933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:43.363106 11933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:43.363118 11933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:43.363131 11933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:43.364593 11933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:38915
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:43.364933 11933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:43.365296 11933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:55:43.371416 11933 server_base.cc:1034] running on GCE node
W20260808 22:55:43.371299 11942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.371351 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.371292 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:43.371937 11933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:43.372331 11933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:43.373569 11933 hybrid_clock.cc:648] HybridClock initialized: now 1786229743373533 us; error 42 us; skew 500 ppm
I20260808 22:55:43.375418 11933 webserver.cc:460] Webserver started at http://127.1.212.62:37451/ using document root <none> and password file <none>
I20260808 22:55:43.375809 11933 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:43.375893 11933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:43.376058 11933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:43.377393 11933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "ea84ee5ef00442059183b2c3ea2c8c6a"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "5e58a15537572eaf25bf228e3d77ca3d"
server_key_iv: "1b326e0fb74fcd3f4bf5295432c436ff"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.377817 11933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "ea84ee5ef00442059183b2c3ea2c8c6a"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "5e58a15537572eaf25bf228e3d77ca3d"
server_key_iv: "1b326e0fb74fcd3f4bf5295432c436ff"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.379359 11933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:43.380347 11949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.380616 11933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:43.380743 11933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "ea84ee5ef00442059183b2c3ea2c8c6a"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "5e58a15537572eaf25bf228e3d77ca3d"
server_key_iv: "1b326e0fb74fcd3f4bf5295432c436ff"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.380904 11933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:43.394429 11933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:43.394824 11933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:43.395005 11933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:43.399538 11933 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:38915
I20260808 22:55:43.399581 12006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:38915 every 8 connection(s)
I20260808 22:55:43.400063 11933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:43.400938 12007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:43.403730 12007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a: Bootstrap starting.
I20260808 22:55:43.404337 12007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:43.404590 12007 log.cc:824] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:43.405427 12007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a: No bootstrap required, opened a new log
I20260808 22:55:43.407051 12007 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } }
I20260808 22:55:43.407202 12007 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:43.407224 12007 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea84ee5ef00442059183b2c3ea2c8c6a, State: Initialized, Role: FOLLOWER
I20260808 22:55:43.407375 12007 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } }
I20260808 22:55:43.407465 12007 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:43.407490 12007 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:43.407521 12007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:43.408083 12007 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } }
I20260808 22:55:43.408174 12007 leader_election.cc:302] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea84ee5ef00442059183b2c3ea2c8c6a; no voters:
I20260808 22:55:43.408319 12007 leader_election.cc:288] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:43.408416 12010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:43.408639 12010 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [term 1 LEADER]: Becoming Leader. State: Replica: ea84ee5ef00442059183b2c3ea2c8c6a, State: Running, Role: LEADER
I20260808 22:55:43.408643 12007 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:43.408856 12010 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } }
I20260808 22:55:43.409315 12012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea84ee5ef00442059183b2c3ea2c8c6a. Latest consensus state: current_term: 1 leader_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } } }
I20260808 22:55:43.409422 12012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:43.409480 12011 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea84ee5ef00442059183b2c3ea2c8c6a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 38915 } } }
I20260808 22:55:43.409566 12011 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:43.409648 12021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:43.410293 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 11933
I20260808 22:55:43.410300 12021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:43.410380 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:43.410604 1872 external_mini_cluster.cc:1428] Setting key 74728b7f1d7d04850f9508a4175de017
I20260808 22:55:43.412180 12021 catalog_manager.cc:1347] Generated new cluster ID: 6b83ebc229fe4d558a28269d04d6f1bb
I20260808 22:55:43.412240 12021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:43.417443 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.411131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.1.212.62:38915)
0808 22:55:43.411337 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.411347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.411545 (+ 198us) server_negotiation.cc:407] Connection header received
0808 22:55:43.411755 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.411763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.411855 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.412007 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.412259 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.413686 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.414819 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.415519 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.415660 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.416415 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:43.416421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:43.416421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:43.416684 (+ 263us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:43.416732 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:43.416748 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:43.416879 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:43.416979 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:55:43.417188 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260808 22:55:43.427161 12021 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:43.427733 12021 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:43.431346 12021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea84ee5ef00442059183b2c3ea2c8c6a: Generated new TSK 0
I20260808 22:55:43.431602 12021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:43.440128 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--builtin_ntp_servers=127.1.212.20:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 22:55:43.528258 12036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:43.528497 12036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:43.528532 12036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:43.529856 12036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:43.529909 12036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:43.529937 12036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:43.531373 12036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:43.531682 12036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:43.532022 12036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:43.537532 12042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.537551 12045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.537523 12043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:43.537846 12036 server_base.cc:1034] running on GCE node
I20260808 22:55:43.538144 12036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:43.538509 12036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:43.539708 12036 hybrid_clock.cc:648] HybridClock initialized: now 1786229743539681 us; error 43 us; skew 500 ppm
I20260808 22:55:43.541313 12036 webserver.cc:460] Webserver started at http://127.1.212.1:34463/ using document root <none> and password file <none>
I20260808 22:55:43.541608 12036 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:43.541676 12036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:43.541823 12036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:43.543073 12036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "357a4cffc5ad4f609b5b97bf6ad990dc"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "a0b1bfe1b90bcca6261f1e8e34c85eec"
server_key_iv: "372ccf836416acab6dbc94598027befc"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.543406 12036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "357a4cffc5ad4f609b5b97bf6ad990dc"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "a0b1bfe1b90bcca6261f1e8e34c85eec"
server_key_iv: "372ccf836416acab6dbc94598027befc"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.544872 12036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 22:55:43.545670 12051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.545853 12036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:43.545962 12036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "357a4cffc5ad4f609b5b97bf6ad990dc"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "a0b1bfe1b90bcca6261f1e8e34c85eec"
server_key_iv: "372ccf836416acab6dbc94598027befc"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.546051 12036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:43.560019 12036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:43.560402 12036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:43.560575 12036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:43.560904 12036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:43.561275 12036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:43.561332 12036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.561375 12036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:43.561404 12036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.567965 12036 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:38465
I20260808 22:55:43.568022 12173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:38465 every 8 connection(s)
I20260808 22:55:43.568377 12036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:43.573916 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.569201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:47835 (local address 127.1.212.62:38915)
0808 22:55:43.569270 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.569272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.569500 (+ 228us) server_negotiation.cc:407] Connection header received
0808 22:55:43.570052 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.570053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.570066 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.570119 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.570834 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.571344 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.572434 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.572629 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.572690 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.573678 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:43.573680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:43.573680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:43.573685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:43.573701 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:43.573705 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:43.573753 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:43.573821 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 22:55:43.573850 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:55:43.574146 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.569189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:38915 (local address 127.1.212.1:47835)
0808 22:55:43.569382 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:43.569417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:43.569939 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:43.570176 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:43.570191 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:43.570200 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:43.570752 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.570757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.571463 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.571464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.572340 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.572343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.572720 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.572721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.572952 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.573519 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:43.573519 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:43.573553 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:43.573581 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:43.573764 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:43.573768 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:43.573771 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:43.573825 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:55:43.573920 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260808 22:55:43.574702 12174 heartbeater.cc:347] Connected to a master server at 127.1.212.62:38915
I20260808 22:55:43.574869 12174 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:43.575073 12174 heartbeater.cc:511] Master 127.1.212.62:38915 requested a full tablet report, sending...
I20260808 22:55:43.575626 11967 ts_manager.cc:194] Registered new tserver with Master: 357a4cffc5ad4f609b5b97bf6ad990dc (127.1.212.1:38465)
I20260808 22:55:43.576277 11967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.1:47835
I20260808 22:55:43.576656 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12036
I20260808 22:55:43.576761 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:43.576980 1872 external_mini_cluster.cc:1428] Setting key 8a9b95cb9321e68c0c3534a41ee274c6
I20260808 22:55:43.577936 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--builtin_ntp_servers=127.1.212.20:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 22:55:43.664695 12177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:43.664973 12177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:43.664997 12177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:43.666314 12177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:43.666373 12177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:43.666402 12177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:43.667919 12177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:43.668256 12177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:43.668638 12177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:43.674064 12184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.674068 12187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:43.674304 12177 server_base.cc:1034] running on GCE node
W20260808 22:55:43.674109 12185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:43.674631 12177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:43.674927 12177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:43.676108 12177 hybrid_clock.cc:648] HybridClock initialized: now 1786229743676091 us; error 39 us; skew 500 ppm
I20260808 22:55:43.677709 12177 webserver.cc:460] Webserver started at http://127.1.212.2:39481/ using document root <none> and password file <none>
I20260808 22:55:43.678017 12177 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:43.678097 12177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:43.678267 12177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:43.679468 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "9956182a07f64c0891624bb3224bfc89"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "3b2e4acf413ada9c535cdc28e94194b0"
server_key_iv: "b4d6a187d97cb8159408d2dc89092fce"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.679883 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "9956182a07f64c0891624bb3224bfc89"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "3b2e4acf413ada9c535cdc28e94194b0"
server_key_iv: "b4d6a187d97cb8159408d2dc89092fce"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.681526 12177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:43.682403 12193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.682619 12177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:43.682735 12177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "9956182a07f64c0891624bb3224bfc89"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "3b2e4acf413ada9c535cdc28e94194b0"
server_key_iv: "b4d6a187d97cb8159408d2dc89092fce"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.682822 12177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:43.696727 12177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:43.697185 12177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:43.697360 12177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:43.697638 12177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:43.698019 12177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:43.698079 12177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.698117 12177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:43.698149 12177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.704782 12177 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:46471
I20260808 22:55:43.704870 12318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:46471 every 8 connection(s)
I20260808 22:55:43.705261 12177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:43.711458 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.706128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:39123 (local address 127.1.212.62:38915)
0808 22:55:43.706209 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.706211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.706460 (+ 249us) server_negotiation.cc:407] Connection header received
0808 22:55:43.706960 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.706961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.706974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.707040 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.707782 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.708379 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.709763 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.710033 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.710107 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.711191 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:43.711192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:43.711192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:43.711198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:43.711215 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:43.711219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:43.711280 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:43.711356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 22:55:43.711388 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:43.711671 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.706084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:38915 (local address 127.1.212.2:39123)
0808 22:55:43.706292 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:43.706337 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:43.706863 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:43.707111 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:43.707115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:43.707123 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:43.707681 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.707686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.708591 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.708593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.709637 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.709642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.710183 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.710184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.710352 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.711017 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:43.711018 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:43.711052 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:43.711081 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:43.711282 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:43.711285 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:43.711288 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:43.711337 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 22:55:43.711438 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260808 22:55:43.712639 12319 heartbeater.cc:347] Connected to a master server at 127.1.212.62:38915
I20260808 22:55:43.712827 12319 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:43.713106 12319 heartbeater.cc:511] Master 127.1.212.62:38915 requested a full tablet report, sending...
I20260808 22:55:43.713740 11967 ts_manager.cc:194] Registered new tserver with Master: 9956182a07f64c0891624bb3224bfc89 (127.1.212.2:46471)
I20260808 22:55:43.714162 11967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.2:39123
I20260808 22:55:43.714600 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12177
I20260808 22:55:43.714707 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:43.714918 1872 external_mini_cluster.cc:1428] Setting key 110460e56b10f0b67976f602c36bbe9a
I20260808 22:55:43.715904 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--builtin_ntp_servers=127.1.212.20:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 22:55:43.806352 12322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:43.806597 12322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:43.806623 12322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:43.807817 12322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:43.807858 12322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:43.807884 12322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:43.809589 12322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:43.809834 12322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:43.810137 12322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:43.815668 12328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.815656 12327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:43.815656 12331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:43.815690 12322 server_base.cc:1034] running on GCE node
I20260808 22:55:43.816133 12322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:43.816421 12322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:43.817629 12322 hybrid_clock.cc:648] HybridClock initialized: now 1786229743817621 us; error 40 us; skew 500 ppm
I20260808 22:55:43.819322 12322 webserver.cc:460] Webserver started at http://127.1.212.3:37781/ using document root <none> and password file <none>
I20260808 22:55:43.819600 12322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:43.819676 12322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:43.819842 12322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:43.821121 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "006f413983ea497395d4ae37c1b68759"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "d16bffaeedb5f95e8ae322841509db14"
server_key_iv: "c260d8418fccbb40ae9882a9eeb7c58b"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.821543 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "006f413983ea497395d4ae37c1b68759"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "d16bffaeedb5f95e8ae322841509db14"
server_key_iv: "c260d8418fccbb40ae9882a9eeb7c58b"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.822988 12322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:43.824028 12338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.824272 12322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:43.824396 12322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "006f413983ea497395d4ae37c1b68759"
format_stamp: "Formatted at 2026-08-08 22:55:43 on dist-test-slave-1x4g"
server_key: "d16bffaeedb5f95e8ae322841509db14"
server_key_iv: "c260d8418fccbb40ae9882a9eeb7c58b"
server_key_version: "encryptionkey@0"
I20260808 22:55:43.824486 12322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:43.840524 12322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:43.841001 12322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:43.841162 12322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:43.841501 12322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:43.842049 12322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:43.842119 12322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.842165 12322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:43.842185 12322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:43.849040 12322 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:36493
I20260808 22:55:43.849084 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:36493 every 8 connection(s)
I20260808 22:55:43.849512 12322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:43.853015 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12322
I20260808 22:55:43.853138 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:43.853330 1872 external_mini_cluster.cc:1428] Setting key fb41d584c79fd374a0c908ae3f23f13e
I20260808 22:55:43.855448 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.850333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:53503 (local address 127.1.212.62:38915)
0808 22:55:43.850381 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.850383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.850662 (+ 279us) server_negotiation.cc:407] Connection header received
0808 22:55:43.851179 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.851180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.851193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.851272 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.852005 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.852522 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.853900 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.854174 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.854266 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.855145 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:43.855146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:43.855147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:43.855152 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:43.855168 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:43.855172 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:43.855239 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:43.855326 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 22:55:43.855363 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:55:43.855708 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:43.850299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:38915 (local address 127.1.212.3:53503)
0808 22:55:43.850528 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:43.850580 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:43.851082 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:43.851335 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:43.851339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:43.851347 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:43.851896 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.851901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.852644 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.852645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.853790 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:43.853794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:43.854286 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:43.854288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:43.854461 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:43.854981 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:43.854981 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:43.855014 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:43.855040 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:43.855239 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:43.855247 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:43.855250 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:55:43.855328 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:55:43.855465 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260808 22:55:43.856225 12461 heartbeater.cc:347] Connected to a master server at 127.1.212.62:38915
I20260808 22:55:43.856328 12461 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:43.856499 12461 heartbeater.cc:511] Master 127.1.212.62:38915 requested a full tablet report, sending...
I20260808 22:55:43.856920 11967 ts_manager.cc:194] Registered new tserver with Master: 006f413983ea497395d4ae37c1b68759 (127.1.212.3:36493)
I20260808 22:55:43.857267 11967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.3:53503
I20260808 22:55:43.865785 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:55:43.868124 12034 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:43.867406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.1.212.62:38915)
0808 22:55:43.867476 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:43.867478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:43.867585 (+ 107us) server_negotiation.cc:407] Connection header received
0808 22:55:43.867647 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:43.867648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:43.867662 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:43.867726 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:43.868025 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37514: BlockingRecv error: recv got EOF from 127.0.0.1:37514 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:55:43.868135 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12036
I20260808 22:55:43.875933 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12177
I20260808 22:55:43.883855 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12322
I20260808 22:55:43.891270 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 11933
2026-08-08T22:55:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (648 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T22:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:43Z Disabled control of system clock
I20260808 22:55:43.923969 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33735
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:55:44.012542 12473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:44.012897 12473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:44.012985 12473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:44.014413 12473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:44.014518 12473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:44.014549 12473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:44.014571 12473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:44.014595 12473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:44.016216 12473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:44.016584 12473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:44.017002 12473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:44.022538 12479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.022531 12478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.022507 12481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:44.022797 12473 server_base.cc:1034] running on GCE node
I20260808 22:55:44.023083 12473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:44.023356 12473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:44.024511 12473 hybrid_clock.cc:648] HybridClock initialized: now 1786229744024488 us; error 39 us; skew 500 ppm
I20260808 22:55:44.026093 12473 webserver.cc:460] Webserver started at http://127.1.212.62:37663/ using document root <none> and password file <none>
I20260808 22:55:44.026387 12473 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:44.026465 12473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:44.026638 12473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:44.027837 12473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "cbd080ff0a6344de99a18d6d8fd7847a"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "312df3d11a5c77c475d7946ba73470d8"
server_key_iv: "1e7a04ece058982d0be1f5b6d3e8ce9e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.028257 12473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "cbd080ff0a6344de99a18d6d8fd7847a"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "312df3d11a5c77c475d7946ba73470d8"
server_key_iv: "1e7a04ece058982d0be1f5b6d3e8ce9e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.029805 12473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:44.030692 12487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.030900 12473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:44.031019 12473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "cbd080ff0a6344de99a18d6d8fd7847a"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "312df3d11a5c77c475d7946ba73470d8"
server_key_iv: "1e7a04ece058982d0be1f5b6d3e8ce9e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.031117 12473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:44.034925 12473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:44.035252 12473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:44.035410 12473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:44.039853 12473 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33735
I20260808 22:55:44.039894 12540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33735 every 8 connection(s)
I20260808 22:55:44.040334 12473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:44.041239 12541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:44.044270 12541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: Bootstrap starting.
I20260808 22:55:44.044997 12541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:44.045272 12541 log.cc:824] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:44.046046 12541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: No bootstrap required, opened a new log
I20260808 22:55:44.047735 12541 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } }
I20260808 22:55:44.047900 12541 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:44.047924 12541 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd080ff0a6344de99a18d6d8fd7847a, State: Initialized, Role: FOLLOWER
I20260808 22:55:44.048092 12541 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } }
I20260808 22:55:44.048182 12541 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:44.048208 12541 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:44.048238 12541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:44.048949 12541 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } }
I20260808 22:55:44.049052 12541 leader_election.cc:302] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbd080ff0a6344de99a18d6d8fd7847a; no voters:
I20260808 22:55:44.049201 12541 leader_election.cc:288] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:44.049299 12545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:44.049499 12545 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [term 1 LEADER]: Becoming Leader. State: Replica: cbd080ff0a6344de99a18d6d8fd7847a, State: Running, Role: LEADER
I20260808 22:55:44.049521 12541 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:44.049561 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12473
I20260808 22:55:44.049649 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:44.049623 12545 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } }
I20260808 22:55:44.049871 1872 external_mini_cluster.cc:1428] Setting key 1b07d9fb30765dee5ffdbe418d1e5af2
I20260808 22:55:44.050002 12545 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbd080ff0a6344de99a18d6d8fd7847a. Latest consensus state: current_term: 1 leader_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } } }
I20260808 22:55:44.050096 12545 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:44.050408 12546 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd080ff0a6344de99a18d6d8fd7847a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33735 } } }
I20260808 22:55:44.050547 12546 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a [sys.catalog]: This master's current role is: LEADER
W20260808 22:55:44.051538 12564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:44.051614 12564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:44.051672 12552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:44.051837 12552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:44.052740 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.050449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.1.212.62:33735)
0808 22:55:44.050754 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:44.050765 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:44.050839 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:55:44.051098 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:44.051106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:44.051156 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:44.051304 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:44.052234 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:44.052238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:44.052238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:44.052243 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:44.052300 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:44.052317 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:44.052395 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:44.052493 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:55:44.052558 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
I20260808 22:55:44.053606 12552 catalog_manager.cc:1347] Generated new cluster ID: 000d83b549ef4ee0a3d54d6513e2f371
I20260808 22:55:44.053668 12552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:44.065235 12552 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:44.065471 12552 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:44.071283 12552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbd080ff0a6344de99a18d6d8fd7847a: Generated new TSK 0
I20260808 22:55:44.071586 12552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:55:44.079347 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--builtin_ntp_servers=127.1.212.20:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:55:44.171972 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:44.172246 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:44.172289 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:44.173695 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:44.173771 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:44.173822 12569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:44.175436 12569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:44.175737 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:44.176074 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:44.182449 12575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.182456 12578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.182459 12576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:44.182912 12569 server_base.cc:1034] running on GCE node
I20260808 22:55:44.183189 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:44.183516 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:44.184700 12569 hybrid_clock.cc:648] HybridClock initialized: now 1786229744184687 us; error 35 us; skew 500 ppm
I20260808 22:55:44.186172 12569 webserver.cc:460] Webserver started at http://127.1.212.1:37447/ using document root <none> and password file <none>
I20260808 22:55:44.186487 12569 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:44.186565 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:44.186734 12569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:44.187896 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "2ed213b15f6f4ed7865c5c87b86e7ba3"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "d65abbd992656273cfd03fa00fa46579"
server_key_iv: "8021caaf9b9c0090fdba910e0f0573f7"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.188323 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "2ed213b15f6f4ed7865c5c87b86e7ba3"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "d65abbd992656273cfd03fa00fa46579"
server_key_iv: "8021caaf9b9c0090fdba910e0f0573f7"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.189924 12569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:44.190801 12584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.191011 12569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:44.191133 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "2ed213b15f6f4ed7865c5c87b86e7ba3"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "d65abbd992656273cfd03fa00fa46579"
server_key_iv: "8021caaf9b9c0090fdba910e0f0573f7"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.191219 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:44.194893 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:44.195199 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:44.195349 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:44.195590 12569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:44.195941 12569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:44.195997 12569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.196040 12569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:44.196066 12569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.202513 12569 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:42467
I20260808 22:55:44.202610 12711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:42467 every 8 connection(s)
I20260808 22:55:44.202987 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:55:44.205876 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12569
I20260808 22:55:44.205996 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:44.206214 1872 external_mini_cluster.cc:1428] Setting key fc7091f3b84f4859e5fa158a258e4f53
I20260808 22:55:44.206802 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.203861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:38025 (local address 127.1.212.62:33735)
0808 22:55:44.203931 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:44.203933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:44.204203 (+ 270us) server_negotiation.cc:407] Connection header received
0808 22:55:44.204808 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:44.204810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:44.204822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:44.204875 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:44.206459 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:44.206461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:44.206462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:44.206471 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:44.206496 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:44.206502 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:44.206589 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:44.206713 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 22:55:44.206733 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:55:44.206996 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.203857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33735 (local address 127.1.212.1:38025)
0808 22:55:44.204076 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:44.204114 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:44.204652 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:44.204959 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:44.204963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:44.204971 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:44.206191 (+ 1220us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:44.206192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:44.206245 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:44.206291 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:44.206593 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:44.206601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:44.206605 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:44.206701 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:55:44.206772 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260808 22:55:44.207418 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--builtin_ntp_servers=127.1.212.20:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 22:55:44.207533 12712 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33735
I20260808 22:55:44.207643 12712 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:44.207810 12712 heartbeater.cc:511] Master 127.1.212.62:33735 requested a full tablet report, sending...
I20260808 22:55:44.208447 12504 ts_manager.cc:194] Registered new tserver with Master: 2ed213b15f6f4ed7865c5c87b86e7ba3 (127.1.212.1:42467)
W20260808 22:55:44.295380 12715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:44.295648 12715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:44.295691 12715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:44.297147 12715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:44.297222 12715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:44.297274 12715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:44.298833 12715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:44.299182 12715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:44.299526 12715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:44.305119 12721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.305119 12720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.305119 12723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:44.305474 12715 server_base.cc:1034] running on GCE node
I20260808 22:55:44.305759 12715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:44.306082 12715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:44.307265 12715 hybrid_clock.cc:648] HybridClock initialized: now 1786229744307236 us; error 44 us; skew 500 ppm
I20260808 22:55:44.308734 12715 webserver.cc:460] Webserver started at http://127.1.212.2:38511/ using document root <none> and password file <none>
I20260808 22:55:44.309134 12715 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:44.309195 12715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:44.309338 12715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:44.310611 12715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "4f3295797a2b48a3903fc7843e735a43"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "13574e788ca28c622eb91557c1f497ad"
server_key_iv: "baa3991952e77d3989a041f4da6355dd"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.311033 12715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "4f3295797a2b48a3903fc7843e735a43"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "13574e788ca28c622eb91557c1f497ad"
server_key_iv: "baa3991952e77d3989a041f4da6355dd"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.312628 12715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:44.313652 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.313870 12715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:44.314011 12715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "4f3295797a2b48a3903fc7843e735a43"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "13574e788ca28c622eb91557c1f497ad"
server_key_iv: "baa3991952e77d3989a041f4da6355dd"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.314122 12715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:44.318493 12715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:44.318857 12715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:44.319032 12715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:44.319293 12715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:44.319676 12715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:44.319744 12715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.319796 12715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:44.319832 12715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.327464 12715 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:34065
I20260808 22:55:44.327555 12851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:34065 every 8 connection(s)
I20260808 22:55:44.328009 12715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:44.331538 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.328992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:38877 (local address 127.1.212.62:33735)
0808 22:55:44.329058 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:44.329061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:44.329318 (+ 257us) server_negotiation.cc:407] Connection header received
0808 22:55:44.329877 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:44.329879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:44.329886 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:44.329946 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:44.331216 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:44.331217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:44.331218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:44.331223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:44.331248 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:44.331253 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:44.331326 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:44.331425 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:55:44.331443 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:55:44.331805 12854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.328977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33735 (local address 127.1.212.2:38877)
0808 22:55:44.329175 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:44.329214 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:44.329754 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:44.330027 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:44.330031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:44.330041 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:44.331037 (+ 996us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:44.331037 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:44.331071 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:44.331104 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:44.331322 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:44.331330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:44.331334 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:44.331436 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 22:55:44.331517 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 22:55:44.332381 12852 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33735
I20260808 22:55:44.332487 12852 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:44.332665 12852 heartbeater.cc:511] Master 127.1.212.62:33735 requested a full tablet report, sending...
I20260808 22:55:44.333113 12504 ts_manager.cc:194] Registered new tserver with Master: 4f3295797a2b48a3903fc7843e735a43 (127.1.212.2:34065)
I20260808 22:55:44.333997 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12715
I20260808 22:55:44.334126 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:44.334321 1872 external_mini_cluster.cc:1428] Setting key 397d6452a688a64804933f7debdebd87
I20260808 22:55:44.335284 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--builtin_ntp_servers=127.1.212.20:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:55:44.426873 12855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:44.427166 12855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:44.427212 12855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:44.428575 12855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:44.428668 12855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:44.428719 12855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:44.430306 12855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:44.430644 12855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:44.431046 12855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:44.436874 12863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.436874 12864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:44.436879 12867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:44.437114 12855 server_base.cc:1034] running on GCE node
I20260808 22:55:44.437413 12855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:44.437719 12855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:44.438898 12855 hybrid_clock.cc:648] HybridClock initialized: now 1786229744438901 us; error 49 us; skew 500 ppm
I20260808 22:55:44.440332 12855 webserver.cc:460] Webserver started at http://127.1.212.3:36493/ using document root <none> and password file <none>
I20260808 22:55:44.440618 12855 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:44.440697 12855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:44.440922 12855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:44.442147 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "0516f2a3a31b4d97b3bb2189fb3f8719"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "9c44dfa9a98c3b3f2d0707ce94023a25"
server_key_iv: "c7a2348a9a5e2fad28a9fddd3336e27e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.442468 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "0516f2a3a31b4d97b3bb2189fb3f8719"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "9c44dfa9a98c3b3f2d0707ce94023a25"
server_key_iv: "c7a2348a9a5e2fad28a9fddd3336e27e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.443856 12855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:44.444708 12873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.445050 12855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:44.445152 12855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "0516f2a3a31b4d97b3bb2189fb3f8719"
format_stamp: "Formatted at 2026-08-08 22:55:44 on dist-test-slave-1x4g"
server_key: "9c44dfa9a98c3b3f2d0707ce94023a25"
server_key_iv: "c7a2348a9a5e2fad28a9fddd3336e27e"
server_key_version: "encryptionkey@0"
I20260808 22:55:44.445284 12855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:44.449448 12855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:44.449831 12855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:44.449995 12855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:44.450255 12855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:44.450634 12855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:44.450668 12855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.450718 12855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:44.450771 12855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:44.457752 12855 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:36005
I20260808 22:55:44.457847 12997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:36005 every 8 connection(s)
I20260808 22:55:44.458248 12855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:55:44.461433 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 12855
I20260808 22:55:44.461580 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:44.461784 1872 external_mini_cluster.cc:1428] Setting key b66ef58383a61115072d2de4be28100f
I20260808 22:55:44.462786 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.459255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:43585 (local address 127.1.212.62:33735)
0808 22:55:44.459352 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:44.459355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:44.459603 (+ 248us) server_negotiation.cc:407] Connection header received
0808 22:55:44.460186 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:44.460187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:44.460196 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:44.460260 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:44.462342 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:44.462344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:44.462345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:55:44.462352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:44.462375 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:55:44.462380 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:44.462450 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:44.462682 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 22:55:44.462702 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:55:44.463059 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:44.459215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33735 (local address 127.1.212.3:43585)
0808 22:55:44.459462 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:44.459504 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:44.460051 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:44.460348 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:44.460352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:44.460361 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:44.462107 (+ 1746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:55:44.462107 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:44.462158 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:55:44.462211 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:44.462564 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:44.462573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:44.462577 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:55:44.462673 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:55:44.462771 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":168,"threads_started":1}
I20260808 22:55:44.463615 12998 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33735
I20260808 22:55:44.463755 12998 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:44.463936 12998 heartbeater.cc:511] Master 127.1.212.62:33735 requested a full tablet report, sending...
I20260808 22:55:44.464432 12504 ts_manager.cc:194] Registered new tserver with Master: 0516f2a3a31b4d97b3bb2189fb3f8719 (127.1.212.3:36005)
I20260808 22:55:44.474068 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:55:44.476290 12556 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:55:44.475685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.1.212.62:33735)
0808 22:55:44.475778 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:44.475781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:44.475880 (+ 99us) server_negotiation.cc:407] Connection header received
0808 22:55:44.475988 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:44.475989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:44.475995 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:44.476049 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:44.476208 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40550: BlockingRecv error: recv got EOF from 127.0.0.1:40550 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:55:44.476361 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12569
I20260808 22:55:44.483888 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12715
I20260808 22:55:44.490813 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12855
I20260808 22:55:44.499845 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 12473
2026-08-08T22:55:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (607 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:44 dist-test-slave-1x4g krb5kdc[13008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:44 dist-test-slave-1x4g krb5kdc[13008](info): set up 2 sockets
Aug 08 22:55:44 dist-test-slave-1x4g krb5kdc[13008](info): commencing operation
krb5kdc: starting...
W20260808 22:55:45.586510 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:45 dist-test-slave-1x4g krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:55:45.727818 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36025
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:36025
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:45.813647 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:45.813899 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:45.813925 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:45.815272 13022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:55:45.815331 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:45.815346 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:45.815359 13022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:55:45.815416 13022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:55:45.817205 13022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:36025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:45.817528 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:45.817795 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:45.823165 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:45.823192 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:45.823172 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:45.823555 13022 server_base.cc:1034] running on GCE node
I20260808 22:55:45.823896 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:45.824290 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:45.825470 13022 hybrid_clock.cc:648] HybridClock initialized: now 1786229745825463 us; error 42 us; skew 500 ppm
Aug 08 22:55:45 dist-test-slave-1x4g krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229745, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:45.828418 13022 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:55:45.828992 13022 webserver.cc:460] Webserver started at http://127.1.212.62:36763/ using document root <none> and password file <none>
I20260808 22:55:45.829264 13022 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:45.829340 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:45.829495 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:45.830660 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "74e0131c4a324b8ebb25e976791ce80b"
format_stamp: "Formatted at 2026-08-08 22:55:45 on dist-test-slave-1x4g"
server_key: "eac03342464ad273cdd97530a9e04c96"
server_key_iv: "cf349948d9af7db3f44a285d1ae319bf"
server_key_version: "encryptionkey@0"
I20260808 22:55:45.831068 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "74e0131c4a324b8ebb25e976791ce80b"
format_stamp: "Formatted at 2026-08-08 22:55:45 on dist-test-slave-1x4g"
server_key: "eac03342464ad273cdd97530a9e04c96"
server_key_iv: "cf349948d9af7db3f44a285d1ae319bf"
server_key_version: "encryptionkey@0"
I20260808 22:55:45.832603 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:55:45.833523 13037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:45.833801 13022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:45.833917 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "74e0131c4a324b8ebb25e976791ce80b"
format_stamp: "Formatted at 2026-08-08 22:55:45 on dist-test-slave-1x4g"
server_key: "eac03342464ad273cdd97530a9e04c96"
server_key_iv: "cf349948d9af7db3f44a285d1ae319bf"
server_key_version: "encryptionkey@0"
I20260808 22:55:45.834002 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:45.844753 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:45.847051 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:45.847240 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:45.851697 13022 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:36025
I20260808 22:55:45.851744 13092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:36025 every 8 connection(s)
I20260808 22:55:45.852160 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:55:45.853433 13093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:45.854162 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 13022
I20260808 22:55:45.854276 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:55:45.854478 1872 external_mini_cluster.cc:1428] Setting key c0ea19686c60f859e7f35f1a83ca66bc
I20260808 22:55:45.857261 13093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: Bootstrap starting.
I20260808 22:55:45.858088 13093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:45.858464 13093 log.cc:824] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:45.859380 13093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: No bootstrap required, opened a new log
Aug 08 22:55:45 dist-test-slave-1x4g krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:45.862645 13093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } }
I20260808 22:55:45.862835 13093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:45.862864 13093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e0131c4a324b8ebb25e976791ce80b, State: Initialized, Role: FOLLOWER
I20260808 22:55:45.863050 13093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } }
I20260808 22:55:45.863188 13093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:55:45.863233 13093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:55:45.863296 13093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:45.864354 13093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } }
I20260808 22:55:45.864506 13093 leader_election.cc:302] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74e0131c4a324b8ebb25e976791ce80b; no voters:
I20260808 22:55:45.864737 13093 leader_election.cc:288] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:55:45.864856 13100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:45.865068 13100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [term 1 LEADER]: Becoming Leader. State: Replica: 74e0131c4a324b8ebb25e976791ce80b, State: Running, Role: LEADER
I20260808 22:55:45.865103 13093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:55:45.865177 13100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } }
I20260808 22:55:45.865566 13100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e0131c4a324b8ebb25e976791ce80b. Latest consensus state: current_term: 1 leader_uuid: "74e0131c4a324b8ebb25e976791ce80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } } }
I20260808 22:55:45.865664 13100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [sys.catalog]: This master's current role is: LEADER
I20260808 22:55:45.866178 13101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74e0131c4a324b8ebb25e976791ce80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e0131c4a324b8ebb25e976791ce80b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36025 } } }
I20260808 22:55:45.866263 13101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b [sys.catalog]: This master's current role is: LEADER
W20260808 22:55:45.867264 13115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:55:45.867331 13115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:55:45.867483 13106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:55:45.867705 13106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:55:45.867954 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:45.855109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.1.212.62:36025)
0808 22:55:45.855321 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:45.855330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:45.855358 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:55:45.855806 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:45.855813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:45.855863 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:45.855999 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:45.856946 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:45.857980 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:45.859134 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:45.859630 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:45.859728 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:45.863002 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:45.863008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:45.863009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:45.863026 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:45.865459 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:45.865947 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:45.865953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:45.865953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:45.866101 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:45.866392 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:45.866393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:45.866394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:45.866742 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:45.866938 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:45.867566 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 22:55:45.867729 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260808 22:55:45.869710 13106 catalog_manager.cc:1347] Generated new cluster ID: 30b3011df28742b68ea5969680147b98
I20260808 22:55:45.869783 13106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:55:45.887461 13106 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:55:45.888249 13106 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:55:45.894464 13106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: Generated new TSK 0
I20260808 22:55:45.894708 13106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:55:45.932224 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:46.020366 13121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:46.020597 13121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:46.020624 13121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:46.021991 13121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:46.022053 13121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:46.022083 13121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:55:46.023725 13121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:46.023967 13121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:46.024264 13121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:46.029592 13130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:46.029625 13128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:46.029592 13126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:46.029932 13121 server_base.cc:1034] running on GCE node
I20260808 22:55:46.030148 13121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:46.030503 13121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:46.031699 13121 hybrid_clock.cc:648] HybridClock initialized: now 1786229746031684 us; error 48 us; skew 500 ppm
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:46.034634 13121 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:55:46.035197 13121 webserver.cc:460] Webserver started at http://127.1.212.1:39835/ using document root <none> and password file <none>
I20260808 22:55:46.035451 13121 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:46.035529 13121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:46.035686 13121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:46.036953 13121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "40ffa3c33c9d4332a37bfd895466f18b"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "9a95b9723b5ea2dce840d6f0b3a7c353"
server_key_iv: "d96221a6e9b3173067faa1974b3878a3"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.037403 13121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "40ffa3c33c9d4332a37bfd895466f18b"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "9a95b9723b5ea2dce840d6f0b3a7c353"
server_key_iv: "d96221a6e9b3173067faa1974b3878a3"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.038991 13121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:55:46.039863 13138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.040071 13121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:46.040184 13121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "40ffa3c33c9d4332a37bfd895466f18b"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "9a95b9723b5ea2dce840d6f0b3a7c353"
server_key_iv: "d96221a6e9b3173067faa1974b3878a3"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.040269 13121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:46.059206 13121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:46.061589 13121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:46.061775 13121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:46.062044 13121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:46.062389 13121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:46.062458 13121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.062506 13121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:46.062532 13121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.069235 13121 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:46223
I20260808 22:55:46.069284 13263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:46223 every 8 connection(s)
I20260808 22:55:46.069650 13121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:46.078922 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 13121
I20260808 22:55:46.079097 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:55:46.079346 1872 external_mini_cluster.cc:1428] Setting key b0bf9358117488f6c26afcda998de979
I20260808 22:55:46.079875 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.070469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51389 (local address 127.1.212.62:36025)
0808 22:55:46.070539 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.070541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.070727 (+ 186us) server_negotiation.cc:407] Connection header received
0808 22:55:46.071269 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.071270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.071281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.071331 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:46.072348 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.072923 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.073920 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.074119 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.074170 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.076619 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:46.076621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:46.076621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:46.076629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:46.078050 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.078532 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.078534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.078534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.078600 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.078981 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.078981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.078982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.079144 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:46.079220 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.079646 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.079783 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:55:46.080068 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.070362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:36025 (local address 127.1.212.1:51389)
0808 22:55:46.070615 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.070655 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.071178 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.071396 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.071399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.072031 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:46.072244 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.072250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.073012 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.073013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.073837 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.073840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.074196 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.074197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.074304 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.074930 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:46.074952 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:46.076500 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:46.078157 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.078167 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.078169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.078427 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.078718 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.078720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.078720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.078904 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.079224 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:46.079235 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:46.079445 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.079595 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.079821 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":115,"threads_started":1}
I20260808 22:55:46.080857 13264 heartbeater.cc:347] Connected to a master server at 127.1.212.62:36025
I20260808 22:55:46.080994 13264 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:46.081199 13264 heartbeater.cc:511] Master 127.1.212.62:36025 requested a full tablet report, sending...
I20260808 22:55:46.081876 13056 ts_manager.cc:194] Registered new tserver with Master: 40ffa3c33c9d4332a37bfd895466f18b (127.1.212.1:46223)
I20260808 22:55:46.082638 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:51389
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:55:46.109054 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:46.197621 13269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:46.197863 13269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:46.197901 13269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:46.199215 13269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:46.199275 13269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:46.199306 13269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:55:46.201128 13269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:46.201444 13269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:46.201718 13269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:46.207466 13274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:46.207603 13269 server_base.cc:1034] running on GCE node
W20260808 22:55:46.207471 13277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:46.207537 13275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:46.208015 13269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:46.208321 13269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:46.209550 13269 hybrid_clock.cc:648] HybridClock initialized: now 1786229746209539 us; error 54 us; skew 500 ppm
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:46.212621 13269 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:55:46.213274 13269 webserver.cc:460] Webserver started at http://127.1.212.2:38317/ using document root <none> and password file <none>
I20260808 22:55:46.213567 13269 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:46.213646 13269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:46.213812 13269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:46.214999 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "7bdc3671ebe245a6bb53c573a08ecf85"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a9b195cfd6b874950bfce6efc0c2ac7f"
server_key_iv: "e921da72965c2cde5d4c61aa03b25f06"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.215409 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "7bdc3671ebe245a6bb53c573a08ecf85"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a9b195cfd6b874950bfce6efc0c2ac7f"
server_key_iv: "e921da72965c2cde5d4c61aa03b25f06"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.217028 13269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:55:46.217954 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.218173 13269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:46.218290 13269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "7bdc3671ebe245a6bb53c573a08ecf85"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a9b195cfd6b874950bfce6efc0c2ac7f"
server_key_iv: "e921da72965c2cde5d4c61aa03b25f06"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.218375 13269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:46.229152 13269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:46.231173 13269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:46.231359 13269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:46.231626 13269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:46.231978 13269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:46.232036 13269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.232075 13269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:46.232121 13269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.238951 13269 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:34797
I20260808 22:55:46.239002 13401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:34797 every 8 connection(s)
I20260808 22:55:46.239375 13269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:55:46.245718 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 13269
I20260808 22:55:46.245839 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:55:46.246062 1872 external_mini_cluster.cc:1428] Setting key 839bbfe5fc925ebf21d6ccc5eae88655
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:46.250257 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.240189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:37431 (local address 127.1.212.62:36025)
0808 22:55:46.240237 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.240239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.240508 (+ 269us) server_negotiation.cc:407] Connection header received
0808 22:55:46.241213 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.241214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.241229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.241277 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:46.242350 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.242878 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.244017 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.244229 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.244311 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.247030 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:46.247031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:46.247032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:46.247041 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:46.248572 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.249148 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.249150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.249150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.249223 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.249516 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.249517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.249517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.249652 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:46.249728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.250054 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.250157 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:55:46.250487 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.240171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:36025 (local address 127.1.212.2:37431)
0808 22:55:46.240371 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.240414 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.241084 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.241388 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.241392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.242012 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:46.242238 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.242244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.242991 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.242992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.243918 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.243920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.244338 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.244339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.244473 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.245083 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:46.245094 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:46.246915 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:46.248694 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.248704 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.248707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.249054 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.249304 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.249305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.249305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.249438 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.249734 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:46.249743 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:46.249905 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.250014 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.250215 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260808 22:55:46.251179 13402 heartbeater.cc:347] Connected to a master server at 127.1.212.62:36025
I20260808 22:55:46.251284 13402 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:46.251580 13402 heartbeater.cc:511] Master 127.1.212.62:36025 requested a full tablet report, sending...
I20260808 22:55:46.252068 13056 ts_manager.cc:194] Registered new tserver with Master: 7bdc3671ebe245a6bb53c573a08ecf85 (127.1.212.2:34797)
I20260808 22:55:46.252476 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:37431
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:55:46.276947 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:55:46.364032 13408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:55:46.364326 13408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:55:46.364373 13408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:55:46.365985 13408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:55:46.366068 13408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:55:46.366122 13408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:55:46.368165 13408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:55:46.368486 13408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:55:46.368755 13408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:55:46.374545 13414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:46.374567 13413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:55:46.374569 13416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:55:46.374912 13408 server_base.cc:1034] running on GCE node
I20260808 22:55:46.375190 13408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:55:46.375478 13408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:55:46.376663 13408 hybrid_clock.cc:648] HybridClock initialized: now 1786229746376659 us; error 41 us; skew 500 ppm
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:55:46.379585 13408 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:55:46.380187 13408 webserver.cc:460] Webserver started at http://127.1.212.3:34367/ using document root <none> and password file <none>
I20260808 22:55:46.380442 13408 fs_manager.cc:359] Metadata directory not provided
I20260808 22:55:46.380515 13408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:55:46.380681 13408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:55:46.381916 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "7847287c27fe48fa8d5566b309bb0e6f"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a7a806ce6fb5fd3f18bb89f951d49205"
server_key_iv: "22d54cb6ea4d978ee24c894c191901c4"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.382349 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "7847287c27fe48fa8d5566b309bb0e6f"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a7a806ce6fb5fd3f18bb89f951d49205"
server_key_iv: "22d54cb6ea4d978ee24c894c191901c4"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.384131 13408 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:55:46.385027 13424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.385226 13408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:55:46.385349 13408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "7847287c27fe48fa8d5566b309bb0e6f"
format_stamp: "Formatted at 2026-08-08 22:55:46 on dist-test-slave-1x4g"
server_key: "a7a806ce6fb5fd3f18bb89f951d49205"
server_key_iv: "22d54cb6ea4d978ee24c894c191901c4"
server_key_version: "encryptionkey@0"
I20260808 22:55:46.385442 13408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:55:46.425164 13408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:55:46.427860 13408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:55:46.428050 13408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:55:46.428339 13408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:55:46.428694 13408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:55:46.428751 13408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.428812 13408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:55:46.428862 13408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:55:46.435235 13408 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:39967
I20260808 22:55:46.435281 13543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:39967 every 8 connection(s)
I20260808 22:55:46.435657 13408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229746, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:55:46.444046 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 13408
I20260808 22:55:46.444146 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:55:46.444357 1872 external_mini_cluster.cc:1428] Setting key 8d822ce4459fd7153291a3d37bfeb82f
I20260808 22:55:46.446208 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.436475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:40693 (local address 127.1.212.62:36025)
0808 22:55:46.436543 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.436546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.436820 (+ 274us) server_negotiation.cc:407] Connection header received
0808 22:55:46.437454 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.437455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.437466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.437522 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:46.438547 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.439049 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.440083 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.440268 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.440322 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.442670 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:46.442672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:46.442672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:46.442681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:46.444521 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.445130 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.445131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.445132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.445201 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.445482 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.445483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.445484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.445619 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:46.445695 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.446010 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.446113 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:55:46.446435 13546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.436451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:36025 (local address 127.1.212.3:40693)
0808 22:55:46.436667 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.436708 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.437355 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.437603 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.437607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.438238 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:55:46.438453 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.438458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.439155 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.439156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.439991 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.439993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.440349 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.440350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.440466 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.441053 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:55:46.441063 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:55:46.442560 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:55:46.444649 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.444662 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.444665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.445039 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.445282 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:55:46.445283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:55:46.445283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:55:46.445408 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:55:46.445700 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:55:46.445710 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:55:46.445868 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.445975 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.446167 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260808 22:55:46.446910 13544 heartbeater.cc:347] Connected to a master server at 127.1.212.62:36025
I20260808 22:55:46.447026 13544 heartbeater.cc:464] Registering TS with master...
I20260808 22:55:46.447194 13544 heartbeater.cc:511] Master 127.1.212.62:36025 requested a full tablet report, sending...
I20260808 22:55:46.447606 13056 ts_manager.cc:194] Registered new tserver with Master: 7847287c27fe48fa8d5566b309bb0e6f (127.1.212.3:39967)
I20260808 22:55:46.447955 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:40693
I20260808 22:55:46.456416 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:55:46.464180 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.457899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.1.212.62:36025)
0808 22:55:46.457966 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.457968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.458015 (+ 47us) server_negotiation.cc:407] Connection header received
0808 22:55:46.458065 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.458066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.458087 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.458153 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:55:46.458736 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.459252 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.460001 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.460197 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.460255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.461579 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:55:46.461580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:55:46.461580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:55:46.461588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:55:46.463084 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.463447 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.463448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.463448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.463504 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:55:46.463679 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:55:46.463679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:55:46.463679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:55:46.463842 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:55:46.463926 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.464020 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.464105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:55:46.466638 13056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:55:46.468395 13056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:55:46.475698 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.471230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.1.212.2:34797)
0808 22:55:46.471412 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.471416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.471530 (+ 114us) server_negotiation.cc:407] Connection header received
0808 22:55:46.471648 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.471651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.471670 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.471753 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:46.472037 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.472866 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.474417 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.475244 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.475333 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.475416 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.475546 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.475602 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260808 22:55:46.475687 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.471115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:34797 (local address 127.0.0.1:40016)
0808 22:55:46.471445 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.471456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.471543 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.471850 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.471851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.471853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:46.471945 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.471947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.472985 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.472986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.474269 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.474272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.475333 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.475334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.475421 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.475438 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.475517 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.475581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260808 22:55:46.475886 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.471001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.1.212.3:39967)
0808 22:55:46.471274 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.471280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.471495 (+ 215us) server_negotiation.cc:407] Connection header received
0808 22:55:46.471554 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.471556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.471570 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.471742 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:46.472087 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.473251 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.474515 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.475244 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.475337 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.475403 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.475736 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.475794 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260808 22:55:46.476053 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.470872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39967 (local address 127.0.0.1:46410)
0808 22:55:46.471269 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.471318 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.471389 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.471844 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.471848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.471850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:46.471949 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.471952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.473372 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.473373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.474415 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.474417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.475442 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.475443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.475525 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.475530 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.475586 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.475630 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":8,"thread_start_us":85,"threads_started":1}
I20260808 22:55:46.476630 13335 tablet_service.cc:1467] Processing CreateTablet for tablet d30224155ec84442b75ff77e5874deea (DEFAULT_TABLE table=test-table [id=7e84339fe6c248ae8dfa1a8653965c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:46.476814 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.470937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46223 (local address 127.0.0.1:60504)
0808 22:55:46.471288 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.471318 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.471389 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.471946 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.471947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.471949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:46.471997 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.471998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.474503 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.474503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.475164 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.475166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.476535 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.476536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.476617 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.476623 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.476674 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.476721 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":103,"threads_started":1}
I20260808 22:55:46.476862 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.471116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.1.212.1:46223)
0808 22:55:46.471716 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.471720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.471730 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:55:46.471770 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.471772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.471785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.471860 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:46.472071 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.473266 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.475266 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.476435 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.476532 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.476619 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.476704 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.476751 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":86,"threads_started":1}
I20260808 22:55:46.477599 13191 tablet_service.cc:1467] Processing CreateTablet for tablet d30224155ec84442b75ff77e5874deea (DEFAULT_TABLE table=test-table [id=7e84339fe6c248ae8dfa1a8653965c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:46.477595 13474 tablet_service.cc:1467] Processing CreateTablet for tablet d30224155ec84442b75ff77e5874deea (DEFAULT_TABLE table=test-table [id=7e84339fe6c248ae8dfa1a8653965c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:55:46.477968 13191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30224155ec84442b75ff77e5874deea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:46.477977 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30224155ec84442b75ff77e5874deea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:46.480304 13335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30224155ec84442b75ff77e5874deea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:55:46.481073 13562 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Bootstrap starting.
I20260808 22:55:46.481492 13563 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: Bootstrap starting.
I20260808 22:55:46.482025 13562 tablet_bootstrap.cc:654] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:46.482194 13563 tablet_bootstrap.cc:654] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:46.482664 13562 log.cc:824] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:46.483021 13563 log.cc:824] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:46.483686 13562 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: No bootstrap required, opened a new log
I20260808 22:55:46.483855 13562 ts_tablet_manager.cc:1397] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:55:46.483893 13566 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Bootstrap starting.
I20260808 22:55:46.484166 13563 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: No bootstrap required, opened a new log
I20260808 22:55:46.484241 13563 ts_tablet_manager.cc:1397] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:55:46.484545 13566 tablet_bootstrap.cc:654] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Neither blocks nor log segments found. Creating new log.
I20260808 22:55:46.485167 13566 log.cc:824] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Log is configured to *not* fsync() on all Append() calls
I20260808 22:55:46.485978 13566 tablet_bootstrap.cc:492] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: No bootstrap required, opened a new log
I20260808 22:55:46.486064 13566 ts_tablet_manager.cc:1397] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:55:46.486125 13562 raft_consensus.cc:348] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.486320 13562 raft_consensus.cc:374] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:46.486373 13562 raft_consensus.cc:729] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7847287c27fe48fa8d5566b309bb0e6f, State: Initialized, Role: FOLLOWER
I20260808 22:55:46.486320 13563 raft_consensus.cc:348] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.486527 13563 raft_consensus.cc:374] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:46.486505 13562 consensus_queue.cc:260] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.486569 13563 raft_consensus.cc:729] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ffa3c33c9d4332a37bfd895466f18b, State: Initialized, Role: FOLLOWER
I20260808 22:55:46.486701 13563 consensus_queue.cc:260] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.486822 13562 ts_tablet_manager.cc:1428] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:55:46.486891 13544 heartbeater.cc:503] Master 127.1.212.62:36025 was elected leader, sending a full tablet report...
I20260808 22:55:46.487367 13563 ts_tablet_manager.cc:1428] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:55:46.487507 13264 heartbeater.cc:503] Master 127.1.212.62:36025 was elected leader, sending a full tablet report...
I20260808 22:55:46.488142 13566 raft_consensus.cc:348] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.488320 13566 raft_consensus.cc:374] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:55:46.488375 13566 raft_consensus.cc:729] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdc3671ebe245a6bb53c573a08ecf85, State: Initialized, Role: FOLLOWER
I20260808 22:55:46.488516 13566 consensus_queue.cc:260] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.488865 13566 ts_tablet_manager.cc:1428] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:55:46.488938 13402 heartbeater.cc:503] Master 127.1.212.62:36025 was elected leader, sending a full tablet report...
W20260808 22:55:46.489948 13403 tablet.cc:2369] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:55:46.494827 13570 raft_consensus.cc:482] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:55:46.494982 13570 raft_consensus.cc:504] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.495416 13570 leader_election.cc:288] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7847287c27fe48fa8d5566b309bb0e6f (127.1.212.3:39967), 40ffa3c33c9d4332a37bfd895466f18b (127.1.212.1:46223)
I20260808 22:55:46.498735 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.495728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:49467 (local address 127.1.212.3:39967)
0808 22:55:46.495785 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.495787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.495798 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:55:46.495830 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.495831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.495846 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.495904 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:46.496092 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.496573 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.497874 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.498317 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.498407 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.498420 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.498606 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.498656 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:55:46.498735 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.495591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39967 (local address 127.1.212.2:49467)
0808 22:55:46.495654 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.495670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.495733 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.495912 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.495914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.495916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:46.495987 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.495989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.496693 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.496694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.497735 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.497740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.498407 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.498408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.498498 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.498508 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.498599 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.498653 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 22:55:46.499120 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30224155ec84442b75ff77e5874deea" candidate_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7847287c27fe48fa8d5566b309bb0e6f" is_pre_election: true
I20260808 22:55:46.499327 13495 raft_consensus.cc:2454] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bdc3671ebe245a6bb53c573a08ecf85 in term 0.
I20260808 22:55:46.499413 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.495591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46223 (local address 127.1.212.2:49455)
0808 22:55:46.495809 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.495819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:55:46.495882 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:55:46.496040 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:55:46.496041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:55:46.496043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:55:46.496118 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.496120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.497111 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.497113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.498139 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:55:46.498143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.499133 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.499134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:55:46.499202 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.499207 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:55:46.499277 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:55:46.499328 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":18,"thread_start_us":126,"threads_started":1}
I20260808 22:55:46.499421 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.495845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:49455 (local address 127.1.212.1:46223)
0808 22:55:46.495915 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.495918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.495932 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:55:46.495965 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.495966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.495978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.496036 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:55:46.496199 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.496702 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.498284 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.498731 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.498829 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.498844 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.499301 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.499354 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:55:46.499665 13288 leader_election.cc:302] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7847287c27fe48fa8d5566b309bb0e6f, 7bdc3671ebe245a6bb53c573a08ecf85; no voters:
I20260808 22:55:46.499815 13570 raft_consensus.cc:2781] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:55:46.499902 13570 raft_consensus.cc:482] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:55:46.499833 13212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30224155ec84442b75ff77e5874deea" candidate_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ffa3c33c9d4332a37bfd895466f18b" is_pre_election: true
I20260808 22:55:46.499931 13570 raft_consensus.cc:3037] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:46.500013 13212 raft_consensus.cc:2454] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bdc3671ebe245a6bb53c573a08ecf85 in term 0.
I20260808 22:55:46.500960 13570 raft_consensus.cc:504] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.501129 13570 leader_election.cc:288] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [CANDIDATE]: Term 1 election: Requested vote from peers 7847287c27fe48fa8d5566b309bb0e6f (127.1.212.3:39967), 40ffa3c33c9d4332a37bfd895466f18b (127.1.212.1:46223)
I20260808 22:55:46.501320 13212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30224155ec84442b75ff77e5874deea" candidate_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ffa3c33c9d4332a37bfd895466f18b"
I20260808 22:55:46.501346 13495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30224155ec84442b75ff77e5874deea" candidate_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7847287c27fe48fa8d5566b309bb0e6f"
I20260808 22:55:46.501406 13212 raft_consensus.cc:3037] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:46.501456 13495 raft_consensus.cc:3037] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:55:46.502244 13495 raft_consensus.cc:2454] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bdc3671ebe245a6bb53c573a08ecf85 in term 1.
I20260808 22:55:46.502307 13212 raft_consensus.cc:2454] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bdc3671ebe245a6bb53c573a08ecf85 in term 1.
I20260808 22:55:46.502487 13288 leader_election.cc:302] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7847287c27fe48fa8d5566b309bb0e6f, 7bdc3671ebe245a6bb53c573a08ecf85; no voters:
I20260808 22:55:46.502592 13570 raft_consensus.cc:2781] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:55:46.502779 13570 raft_consensus.cc:686] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 1 LEADER]: Becoming Leader. State: Replica: 7bdc3671ebe245a6bb53c573a08ecf85, State: Running, Role: LEADER
I20260808 22:55:46.502895 13570 consensus_queue.cc:237] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } }
I20260808 22:55:46.503860 13055 catalog_manager.cc:5495] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bdc3671ebe245a6bb53c573a08ecf85 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bdc3671ebe245a6bb53c573a08ecf85" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 34797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260808 22:55:46.513474 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.510860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.1.212.2:34797)
0808 22:55:46.510927 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:55:46.510929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:55:46.510938 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:55:46.510985 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:55:46.511005 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:55:46.511028 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:55:46.511108 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:55:46.511279 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.511778 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.512574 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:55:46.512766 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:55:46.512886 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:55:46.513073 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:55:46.513194 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:55:46.513287 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:55:46.513328 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 22:55:46.513351 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:55:46.515535 13212 raft_consensus.cc:1261] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 1 FOLLOWER]: Refusing update from remote peer 7bdc3671ebe245a6bb53c573a08ecf85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:46.515544 13495 raft_consensus.cc:1261] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 1 FOLLOWER]: Refusing update from remote peer 7bdc3671ebe245a6bb53c573a08ecf85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:55:46.515938 13572 consensus_queue.cc:1035] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40ffa3c33c9d4332a37bfd895466f18b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:46.516098 13570 consensus_queue.cc:1035] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7847287c27fe48fa8d5566b309bb0e6f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:55:46.519277 13578 mvcc.cc:205] Tried to move back new op lower bound from 7316397041724555264 to 7316397041676701696. Current Snapshot: MvccSnapshot[applied={T|T < 7316397041724555264}]
I20260808 22:55:46.520139 13576 mvcc.cc:205] Tried to move back new op lower bound from 7316397041724555264 to 7316397041676701696. Current Snapshot: MvccSnapshot[applied={T|T < 7316397041724555264 or (T in {7316397041724555264})}]
I20260808 22:55:46.521587 13055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:46.521701 13055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:55:46.522485 13055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74e0131c4a324b8ebb25e976791ce80b: Sending DeleteTablet for 3 replicas of tablet d30224155ec84442b75ff77e5874deea
I20260808 22:55:46.522853 13474 tablet_service.cc:1514] Processing DeleteTablet for tablet d30224155ec84442b75ff77e5874deea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:46 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:46410
I20260808 22:55:46.522917 13335 tablet_service.cc:1514] Processing DeleteTablet for tablet d30224155ec84442b75ff77e5874deea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:46 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:40016
I20260808 22:55:46.523121 13586 tablet_replica.cc:330] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: stopping tablet replica
I20260808 22:55:46.523139 13191 tablet_service.cc:1514] Processing DeleteTablet for tablet d30224155ec84442b75ff77e5874deea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:55:46 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:60504
I20260808 22:55:46.523236 13586 raft_consensus.cc:2229] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:46.523399 13588 tablet_replica.cc:330] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b: stopping tablet replica
I20260808 22:55:46.523518 13588 raft_consensus.cc:2229] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:55:46.523527 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 13121
I20260808 22:55:46.523626 13588 raft_consensus.cc:2258] T d30224155ec84442b75ff77e5874deea P 40ffa3c33c9d4332a37bfd895466f18b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:46.528285 13586 raft_consensus.cc:2258] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:46.528579 13587 tablet_replica.cc:330] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: stopping tablet replica
I20260808 22:55:46.528712 13587 raft_consensus.cc:2229] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:55:46.528853 13587 raft_consensus.cc:2258] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:55:46.529073 13586 ts_tablet_manager.cc:1905] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:46.531133 13586 ts_tablet_manager.cc:1918] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:55:46.531221 13586 log.cc:1196] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/d30224155ec84442b75ff77e5874deea
I20260808 22:55:46.531523 13586 ts_tablet_manager.cc:1939] T d30224155ec84442b75ff77e5874deea P 7847287c27fe48fa8d5566b309bb0e6f: Deleting consensus metadata
I20260808 22:55:46.531847 13587 ts_tablet_manager.cc:1905] T d30224155ec84442b75ff77e5874deea P 7bdc3671ebe245a6bb53c573a08ecf85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:55:46.532389 13041 catalog_manager.cc:4841] TS 7847287c27fe48fa8d5566b309bb0e6f (127.1.212.3:39967): tablet d30224155ec84442b75ff77e5874deea (table test-table [id=7e84339fe6c248ae8dfa1a8653965c90]) successfully deleted
W20260808 22:55:46.532816 13038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:46223 (error 108)
I20260808 22:55:46.533174 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 13269
I20260808 22:55:46.534904 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.534706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46223 (local address 127.0.0.1:60516)
0808 22:55:46.534818 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.534864 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260808 22:55:46.535063 13038 catalog_manager.cc:4584] TS 40ffa3c33c9d4332a37bfd895466f18b (127.1.212.1:46223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d30224155ec84442b75ff77e5874deea: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46223: connect: Connection refused (error 111)
I20260808 22:55:46.539777 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 13408
I20260808 22:55:46.539919 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:55:46.539747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:34797 (local address 127.0.0.1:40034)
0808 22:55:46.539822 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 22:55:46.539860 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.2:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 22:55:46.546535 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 13022
2026-08-08T22:55:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2048 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13592](info): set up 2 sockets
Aug 08 22:55:46 dist-test-slave-1x4g krb5kdc[13592](info): commencing operation
krb5kdc: starting...
W20260808 22:55:47.626235 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:55:47 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:55:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangeradmin_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangeradmin_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangerlookup_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangerlookup_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
I20260808 22:55:47.816167 1872 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskusC_hJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/postgres -l logfile start
2026-08-08 22:55:50.617 UTC [13622] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:55:50.618 UTC [13622] LOG: listening on IPv4 address "127.1.212.20", port 46171
2026-08-08 22:55:50.622 UTC [13622] LOG: listening on Unix socket "/tmp/.s.PGSQL.46171"
2026-08-08 22:55:50.647 UTC [13625] LOG: database system was shut down at 2026-08-08 22:55:48 UTC
2026-08-08 22:55:50.651 UTC [13622] LOG: database system is ready to accept connections
I20260808 22:55:51.641490 1872 mini_postgres.cc:96] Postgres bound to 46171
2026-08-08 22:55:51.650 UTC [13634] FATAL: database "slave" does not exist
127.1.212.20:46171 - accepting connections
I20260808 22:55:51.651014 1872 mini_ranger.cc:161] Starting Ranger...
I20260808 22:55:51.668937 1872 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:55:52.223706 1872 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:55:52.223913 1872 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin
2026-08-08 22:55:52,543 [I] DB FLAVOR :POSTGRES
2026-08-08 22:55:52,544 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:55:52,544 [I] Checking connection..
2026-08-08 22:55:52,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:55:52,814 [I] Checking connection passed.
2026-08-08 22:55:52,814 [I] --------- Verifying version history table ---------
2026-08-08 22:55:52,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:55:53,107 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:55:53,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:55:53,362 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:55:53,363 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:55:53,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:55:53.621 UTC [13702] WARNING: there is no transaction in progress
2026-08-08 22:55:53,629 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:55:53,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:55:53,915 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:55:53,915 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:55:53,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:55:54,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:55:54,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-08 22:55:54,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:55:54,953 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:55:54,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:55:55,218 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:55:55,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:55:55,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:55:55,723 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:55:55,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:55:56.839 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:56.857 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:56.876 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:56.895 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:56.917 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:56.940 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.004 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.012 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.029 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.040 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.052 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.069 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.081 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.089 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.096 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57.104 UTC [13815] WARNING: there is no transaction in progress
2026-08-08 22:55:57,838 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:55:57,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:55:58,110 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:55:58,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:55:58,384 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:55:58,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:55:58,651 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:55:58,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:55:58,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:55:59,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-08 22:55:59,406 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:55:59,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:55:59,700 [I] DB_PATCHES have already been applied
I20260808 22:55:59.707100 1872 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskusC_hJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:55:59.707178 1872 mini_ranger.cc:193] Using host: 127.1.212.20
I20260808 22:55:59.711340 1872 mini_ranger.cc:227] Ranger admin URL: http://127.1.212.20:33547
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:56:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:56:00.857898 1872 mini_ranger.cc:160] Time spent starting Ranger: real 9.207s user 0.001s sys 0.012s
Aug 08, 2026 10:56:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:56:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:56:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:56:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:56:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:56:24.798336 1872 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangerkms_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/rangerkms_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.212.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.212.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:56:24 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229784, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:56:24.900399 1872 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:56:24.912139 1872 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:56:25.849340 1872 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:56:25.849435 1872 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms
I20260808 22:56:25.849455 1872 mini_ranger_kms.cc:227] Using postgres at 127.1.212.20:46171
2026-08-08 22:56:26,104 [I] DB FLAVOR :POSTGRES
2026-08-08 22:56:26,105 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:56:26,105 [I] Checking connection
2026-08-08 22:56:26,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:56:26,376 [I] connection success
2026-08-08 22:56:26,377 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:56:26,377 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:56:26,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:56:26,661 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:56:26,661 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:56:26,661 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:56:26,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:46171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:56:26,963 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:56:27.496230 1872 mini_ranger_kms.cc:310] Created kms service
I20260808 22:56:27.731617 1872 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:56:27.779606 1872 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:56:27 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229784, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:56:28.238834 1872 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:56:28.238967 1872 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms:/tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:56:28.239009 1872 mini_ranger_kms.cc:240] Using host: 127.1.212.20
I20260808 22:56:28.243450 1872 mini_ranger_kms.cc:276] Ranger KMS PID: 14005
I20260808 22:56:28.243608 1872 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.212.20:34989
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:56:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 22:56:30.651139 1872 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.751s user 0.006s sys 0.231s
I20260808 22:56:30.651271 1872 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:56:30.651371 1872 mini_ranger_kms.cc:401] 127.1.212.20:34989/kms/v1/keys
Aug 08 22:56:30 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786229790, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 22:56:31 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229790, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
Aug 08 22:56:31 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229790, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
Aug 08 22:56:33 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:56:34.028499 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39289
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:39289
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:34989/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:56:34.119851 14058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:56:34.120115 14058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:56:34.120141 14058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:56:34.121503 14058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:56:34.121553 14058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:56:34.121568 14058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:56:34.121580 14058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:56:34.121592 14058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:56:34.123418 14058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.212.20:34989/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:39289
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:56:34.123710 14058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:56:34.123966 14058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:56:34.129632 14058 server_base.cc:1034] running on GCE node
W20260808 22:56:34.129765 14065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:34.129603 14064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:34.129602 14067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:56:34.130189 14058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:56:34.130534 14058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:56:34.131718 14058 hybrid_clock.cc:648] HybridClock initialized: now 1786229794131705 us; error 37 us; skew 500 ppm
Aug 08 22:56:34 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:56:34.134558 14058 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:56:34.135131 14058 webserver.cc:460] Webserver started at http://127.1.212.62:37217/ using document root <none> and password file <none>
I20260808 22:56:34.135390 14058 fs_manager.cc:359] Metadata directory not provided
I20260808 22:56:34.135432 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:56:34.135571 14058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:56:34 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:56:34.203859 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "124e5d9e9c6f4dabaa700f1ca4627be1"
format_stamp: "Formatted at 2026-08-08 22:56:34 on dist-test-slave-1x4g"
server_key: "569b1227dcda0487b53099a48f041fe8"
server_key_iv: "79a4c3440c01d0abf33e052566d15d10"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:34.204303 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "124e5d9e9c6f4dabaa700f1ca4627be1"
format_stamp: "Formatted at 2026-08-08 22:56:34 on dist-test-slave-1x4g"
server_key: "569b1227dcda0487b53099a48f041fe8"
server_key_iv: "79a4c3440c01d0abf33e052566d15d10"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:34.205993 14058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:56:34.221868 14075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:34.222213 14058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:56:34.222347 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "124e5d9e9c6f4dabaa700f1ca4627be1"
format_stamp: "Formatted at 2026-08-08 22:56:34 on dist-test-slave-1x4g"
server_key: "569b1227dcda0487b53099a48f041fe8"
server_key_iv: "79a4c3440c01d0abf33e052566d15d10"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:34.222479 14058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:56:34.243427 14058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:56:34.245983 14058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:56:34.246181 14058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:56:34.252761 14058 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:39289
I20260808 22:56:34.252861 14130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:39289 every 8 connection(s)
I20260808 22:56:34.253360 14058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:56:34.254213 14131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:56:34.256740 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 14058
I20260808 22:56:34.256902 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:56:34.257058 14131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: Bootstrap starting.
I20260808 22:56:34.257085 1872 external_mini_cluster.cc:1428] Setting key 7cb1380df6f02ead9f1ab38ea52e35c2
I20260808 22:56:34.258282 14131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: Neither blocks nor log segments found. Creating new log.
I20260808 22:56:34.258674 14131 log.cc:824] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:56:34.259537 14131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: No bootstrap required, opened a new log
I20260808 22:56:34.262200 14131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } }
I20260808 22:56:34.262398 14131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:56:34.262470 14131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124e5d9e9c6f4dabaa700f1ca4627be1, State: Initialized, Role: FOLLOWER
I20260808 22:56:34.262625 14131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } }
I20260808 22:56:34.262720 14131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:56:34.262754 14131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:56:34.262820 14131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:56:34.263674 14131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } }
I20260808 22:56:34.263823 14131 leader_election.cc:302] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 124e5d9e9c6f4dabaa700f1ca4627be1; no voters:
I20260808 22:56:34.264083 14131 leader_election.cc:288] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:56:34.264148 14137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:56:34.264429 14137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [term 1 LEADER]: Becoming Leader. State: Replica: 124e5d9e9c6f4dabaa700f1ca4627be1, State: Running, Role: LEADER
I20260808 22:56:34.264546 14131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:56:34.264599 14137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } }
I20260808 22:56:34.265190 14139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } } }
I20260808 22:56:34.265272 14139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:56:34.265548 14138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 124e5d9e9c6f4dabaa700f1ca4627be1. Latest consensus state: current_term: 1 leader_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124e5d9e9c6f4dabaa700f1ca4627be1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39289 } } }
I20260808 22:56:34.265653 14138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1 [sys.catalog]: This master's current role is: LEADER
Aug 08 22:56:34 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:56:34.266993 14141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:56:34.267023 14131 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:56:34.267700 14141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:56:34.276089 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:34.257719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.1.212.62:39289)
0808 22:56:34.257942 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:34.257954 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:34.257992 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:56:34.258707 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:34.258717 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:34.258791 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:34.258955 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:56:34.259765 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:34.261333 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:34.262615 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:34.263309 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:34.263435 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:34.266818 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:56:34.266823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:56:34.266823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:56:34.266838 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:56:34.271738 (+ 4900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:34.272377 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:34.272382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:34.272383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:34.272634 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:34.272920 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:34.272921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:34.272922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:34.273627 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:56:34.273878 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:34.275283 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0808 22:56:34.275651 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":117,"threads_started":1}
I20260808 22:56:34.276365 14141 catalog_manager.cc:1347] Generated new cluster ID: 3e3ce583b81d4c6ca1977e1c45b3e45a
I20260808 22:56:34.277925 14141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:56:34.303773 14141 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:56:34.304931 14141 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:56:34.317451 14141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: Generated new TSK 0
I20260808 22:56:34.317843 14141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:56:35.088282 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:34989/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:56:35.189193 14180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:56:35.189435 14180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:56:35.189462 14180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:56:35.190817 14180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:56:35.190883 14180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:56:35.190914 14180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:56:35.192700 14180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.212.20:34989/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:56:35.193087 14180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:56:35.193375 14180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:56:35.200065 14186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.200084 14187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.200424 14189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:56:35.201293 14180 server_base.cc:1034] running on GCE node
I20260808 22:56:35.201551 14180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:56:35.201851 14180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:56:35.203049 14180 hybrid_clock.cc:648] HybridClock initialized: now 1786229795203006 us; error 56 us; skew 500 ppm
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:56:35.206945 14180 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:56:35.207640 14180 webserver.cc:460] Webserver started at http://127.1.212.1:44301/ using document root <none> and password file <none>
I20260808 22:56:35.207966 14180 fs_manager.cc:359] Metadata directory not provided
I20260808 22:56:35.208073 14180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:56:35.208385 14180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:56:35.226475 14180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "a78e5ef6eb90400db00a4207d0460556"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "f6770ddec3e08f17346ccafea9c0e60a"
server_key_iv: "021d7bb1e312a23368e3cbe3861f933f"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.227035 14180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "a78e5ef6eb90400db00a4207d0460556"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "f6770ddec3e08f17346ccafea9c0e60a"
server_key_iv: "021d7bb1e312a23368e3cbe3861f933f"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.229359 14180 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:56:35.248469 14196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.248759 14180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:56:35.248907 14180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "a78e5ef6eb90400db00a4207d0460556"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "f6770ddec3e08f17346ccafea9c0e60a"
server_key_iv: "021d7bb1e312a23368e3cbe3861f933f"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.249053 14180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:56:35.277294 14180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:56:35.279775 14180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:56:35.279992 14180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:56:35.280282 14180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:56:35.280691 14180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:56:35.280751 14180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.280833 14180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:56:35.280874 14180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.290059 14180 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:44011
I20260808 22:56:35.290596 14180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:56:35.291445 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 14180
I20260808 22:56:35.291561 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:56:35.291800 1872 external_mini_cluster.cc:1428] Setting key dc5d27f4e9caa53d1e46e0d483eacc20
I20260808 22:56:35.301097 14318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:44011 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
I20260808 22:56:35.317090 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.301892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39289 (local address 127.1.212.1:36073)
0808 22:56:35.305097 (+ 3205us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.305184 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.305920 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.306347 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.306352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.306569 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:56:35.306800 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.306804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.307665 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.307666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.308691 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.308695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.309349 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.309350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.309541 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.310285 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:56:35.310297 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:56:35.312472 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:56:35.314915 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.314929 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.314933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.315340 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.315619 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.315620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.315620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.315801 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.316153 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:56:35.316163 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:56:35.316403 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.316529 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.316753 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3137,"thread_start_us":118,"threads_started":1}
I20260808 22:56:35.317503 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.301897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:36073 (local address 127.1.212.62:39289)
0808 22:56:35.304969 (+ 3072us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.304972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.306080 (+ 1108us) server_negotiation.cc:407] Connection header received
0808 22:56:35.306113 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.306114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.306130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.306219 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:56:35.306953 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.307486 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.308860 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.309173 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.309227 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.312640 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:56:35.312643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:56:35.312643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:56:35.312654 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:56:35.314779 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.315449 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.315451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.315451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.315528 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.315877 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.315878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.315878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.316041 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:56:35.316121 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.317288 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.317406 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3020,"thread_start_us":135,"threads_started":1}
I20260808 22:56:35.317991 14320 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39289
I20260808 22:56:35.318112 14320 heartbeater.cc:464] Registering TS with master...
I20260808 22:56:35.318341 14320 heartbeater.cc:511] Master 127.1.212.62:39289 requested a full tablet report, sending...
I20260808 22:56:35.318974 14092 ts_manager.cc:194] Registered new tserver with Master: a78e5ef6eb90400db00a4207d0460556 (127.1.212.1:44011)
I20260808 22:56:35.319923 14092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:36073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:56:35.341636 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:34989/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:56:35.477371 14327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:56:35.477725 14327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:56:35.477777 14327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:56:35.479816 14327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:56:35.479890 14327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:56:35.479943 14327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:56:35.483287 14327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.212.20:34989/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:56:35.483590 14327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:56:35.483860 14327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:56:35.490996 14337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.491020 14334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.493048 14335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:56:35.494491 14327 server_base.cc:1034] running on GCE node
I20260808 22:56:35.494788 14327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:56:35.495126 14327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:56:35.496330 14327 hybrid_clock.cc:648] HybridClock initialized: now 1786229795496310 us; error 51 us; skew 500 ppm
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:56:35.500154 14327 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:56:35.500808 14327 webserver.cc:460] Webserver started at http://127.1.212.2:45925/ using document root <none> and password file <none>
I20260808 22:56:35.501129 14327 fs_manager.cc:359] Metadata directory not provided
I20260808 22:56:35.501200 14327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:56:35.501346 14327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:56:35.518900 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "825d189785f14a44926ba2b66e34577b"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "4fae5ca29020bdfe80ab300a5859f0a0"
server_key_iv: "4e4eec4e2f02e155ee5d07d449cfe6f6"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.519757 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "825d189785f14a44926ba2b66e34577b"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "4fae5ca29020bdfe80ab300a5859f0a0"
server_key_iv: "4e4eec4e2f02e155ee5d07d449cfe6f6"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.531665 14327 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.002s sys 0.000s
I20260808 22:56:35.548990 14344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.549280 14327 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 22:56:35.549408 14327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "825d189785f14a44926ba2b66e34577b"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "4fae5ca29020bdfe80ab300a5859f0a0"
server_key_iv: "4e4eec4e2f02e155ee5d07d449cfe6f6"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.549546 14327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:56:35.561261 14327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:56:35.563822 14327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:56:35.564014 14327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:56:35.564277 14327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:56:35.564677 14327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:56:35.564733 14327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.564795 14327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:56:35.564821 14327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.572481 14327 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:35521
I20260808 22:56:35.573025 14327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:56:35.577548 14465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:35521 every 8 connection(s)
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:56:35.580715 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 14327
I20260808 22:56:35.580844 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:56:35.581048 1872 external_mini_cluster.cc:1428] Setting key 65847688ba0a97d4aa811a207273da8a
I20260808 22:56:35.584919 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.574325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:54283 (local address 127.1.212.62:39289)
0808 22:56:35.574374 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.574376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.574389 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:56:35.575178 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.575180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.575193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.575264 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:56:35.575901 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.576747 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.578217 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.578553 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.578639 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.581370 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:56:35.581372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:56:35.581372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:56:35.581382 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:56:35.583046 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.583502 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.583503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.583504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.583573 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.583960 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.583961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.583962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.584131 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:56:35.584180 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.584658 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.584816 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
I20260808 22:56:35.593102 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.573920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39289 (local address 127.1.212.2:54283)
0808 22:56:35.574227 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.574284 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.575039 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.575353 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.575357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.575591 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:56:35.575801 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.575806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.576913 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.576914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.578117 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.578120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.578661 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.578663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.578848 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.579447 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:56:35.579459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:56:35.581176 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:56:35.583138 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.583148 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.583151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.583416 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.583646 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.583647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.583648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.583835 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.584234 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:56:35.584242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:56:35.584469 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.584596 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.584820 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":183,"threads_started":1}
I20260808 22:56:35.593851 14466 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39289
I20260808 22:56:35.593983 14466 heartbeater.cc:464] Registering TS with master...
I20260808 22:56:35.594240 14466 heartbeater.cc:511] Master 127.1.212.62:39289 requested a full tablet report, sending...
I20260808 22:56:35.594852 14092 ts_manager.cc:194] Registered new tserver with Master: 825d189785f14a44926ba2b66e34577b (127.1.212.2:35521)
I20260808 22:56:35.595431 14092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:54283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:56:35.617022 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:34989/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:56:35.739285 14472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:56:35.739667 14472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:56:35.739741 14472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:56:35.741707 14472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:56:35.741845 14472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:56:35.741925 14472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:56:35.743921 14472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.212.20:34989/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:56:35.744228 14472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:56:35.744498 14472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:56:35.750288 14478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.750298 14480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:56:35.750348 14477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:56:35.750705 14472 server_base.cc:1034] running on GCE node
I20260808 22:56:35.751008 14472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:56:35.751324 14472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:56:35.752506 14472 hybrid_clock.cc:648] HybridClock initialized: now 1786229795752488 us; error 34 us; skew 500 ppm
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:56:35.755846 14472 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:56:35.756388 14472 webserver.cc:460] Webserver started at http://127.1.212.3:35773/ using document root <none> and password file <none>
I20260808 22:56:35.756655 14472 fs_manager.cc:359] Metadata directory not provided
I20260808 22:56:35.756732 14472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:56:35.756928 14472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:56:35.772712 14472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "28ec2f457fa4456684c5dd31bcfe5afe"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "5df31bbb2a746a8de450f016323c14dc"
server_key_iv: "0ee4649bb68e4b845d785e0a3946b4c3"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.773380 14472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "28ec2f457fa4456684c5dd31bcfe5afe"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "5df31bbb2a746a8de450f016323c14dc"
server_key_iv: "0ee4649bb68e4b845d785e0a3946b4c3"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.781360 14472 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260808 22:56:35.795898 14488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.796159 14472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:56:35.796271 14472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "28ec2f457fa4456684c5dd31bcfe5afe"
format_stamp: "Formatted at 2026-08-08 22:56:35 on dist-test-slave-1x4g"
server_key: "5df31bbb2a746a8de450f016323c14dc"
server_key_iv: "0ee4649bb68e4b845d785e0a3946b4c3"
server_key_version: "kuduclusterkey@0"
I20260808 22:56:35.796381 14472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:56:35.837191 14472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:56:35.839741 14472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:56:35.839927 14472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:56:35.840217 14472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:56:35.840637 14472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:56:35.840696 14472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.840737 14472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:56:35.840764 14472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:56:35.849879 14472 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:43193
I20260808 22:56:35.850402 14472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:56:35.855752 14613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:43193 every 8 connection(s)
I20260808 22:56:35.856459 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 14472
I20260808 22:56:35.856608 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:56:35.856864 1872 external_mini_cluster.cc:1428] Setting key 77d93191005e40a7ce7ada3c18163ef6
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:56:35.872233 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.857181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34189 (local address 127.1.212.62:39289)
0808 22:56:35.857237 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.857240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.861144 (+ 3904us) server_negotiation.cc:407] Connection header received
0808 22:56:35.861896 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.861898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.861915 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.861981 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:56:35.862630 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.863397 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.865041 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.865351 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.865437 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.868305 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:56:35.868308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:56:35.868308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:56:35.868317 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:56:35.870309 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.870961 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.870963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.870963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.871036 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.871361 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.871362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.871362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.871529 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:56:35.871613 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.872032 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.872131 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:56:35.872601 14616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.856801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:39289 (local address 127.1.212.3:34189)
0808 22:56:35.860941 (+ 4140us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.861025 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.861767 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.862073 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.862077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.862301 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:56:35.862514 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.862519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.863512 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.863514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.864917 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.864923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.865453 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.865455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.865628 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.866347 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:56:35.866360 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:56:35.868168 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:56:35.870414 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.870425 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.870427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.870857 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.871118 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:56:35.871118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:56:35.871119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:56:35.871273 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:56:35.871628 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:56:35.871636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:56:35.871840 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.871973 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.872164 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4079,"thread_start_us":113,"threads_started":1}
I20260808 22:56:35.873147 14614 heartbeater.cc:347] Connected to a master server at 127.1.212.62:39289
I20260808 22:56:35.873276 14614 heartbeater.cc:464] Registering TS with master...
I20260808 22:56:35.873512 14614 heartbeater.cc:511] Master 127.1.212.62:39289 requested a full tablet report, sending...
I20260808 22:56:35.873971 14092 ts_manager.cc:194] Registered new tserver with Master: 28ec2f457fa4456684c5dd31bcfe5afe (127.1.212.3:43193)
I20260808 22:56:35.874467 14092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:34189
I20260808 22:56:35.879765 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:56:35.890455 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.881768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.1.212.62:39289)
0808 22:56:35.881826 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.881828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.882032 (+ 204us) server_negotiation.cc:407] Connection header received
0808 22:56:35.882072 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.882074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.882089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.882159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:56:35.882956 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.883711 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.884851 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.885193 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.885274 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.887108 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:56:35.887109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:56:35.887110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:56:35.887119 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:56:35.889002 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.889469 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.889471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.889471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.889552 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:56:35.889817 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:56:35.889818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:56:35.889818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:56:35.889977 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:56:35.890088 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.890248 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.890348 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:56:35.892622 14092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:56:35.895149 14092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:56:35.904883 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.898614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:44011 (local address 127.0.0.1:44382)
0808 22:56:35.900412 (+ 1798us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.900423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.900485 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.900752 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.900753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.900755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:56:35.900845 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.900847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.902172 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.902173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.903325 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.903329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.904490 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.904491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.904590 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.904606 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.904682 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.904742 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1768,"thread_start_us":93,"threads_started":1}
I20260808 22:56:35.905326 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.898689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.1.212.2:35521)
0808 22:56:35.899478 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.899492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.899737 (+ 245us) server_negotiation.cc:407] Connection header received
0808 22:56:35.899796 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.899798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.899817 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.899910 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:56:35.900203 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.901777 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.903800 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.904623 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.904699 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.904806 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.905181 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.905226 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":99,"threads_started":1}
I20260808 22:56:35.905608 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.899297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.1.212.1:44011)
0808 22:56:35.900202 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.900207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.900547 (+ 340us) server_negotiation.cc:407] Connection header received
0808 22:56:35.900588 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.900590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.900607 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.900674 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:56:35.900934 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.902061 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.903449 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.904278 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.904359 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.904443 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.905331 (+ 888us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.905529 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":58,"threads_started":1}
I20260808 22:56:35.905094 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.898364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35521 (local address 127.0.0.1:45450)
0808 22:56:35.899518 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.899567 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.899649 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.900025 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.900029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.900032 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:56:35.900112 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.900114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.901873 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.901874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.903274 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.903277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.904843 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.904844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.904909 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.904916 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.904964 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.905013 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":94,"threads_started":1}
I20260808 22:56:35.906553 14248 tablet_service.cc:1467] Processing CreateTablet for tablet bf14e8c439764c86a8363922cc3baef0 (DEFAULT_TABLE table=test-table [id=37713b65e9bc44fdbb32c9ee86e7c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:56:35.906746 14395 tablet_service.cc:1467] Processing CreateTablet for tablet bf14e8c439764c86a8363922cc3baef0 (DEFAULT_TABLE table=test-table [id=37713b65e9bc44fdbb32c9ee86e7c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:56:35.907003 14248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf14e8c439764c86a8363922cc3baef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:56:35.907105 14395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf14e8c439764c86a8363922cc3baef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:56:35.908209 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.898971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33502 (local address 127.1.212.3:43193)
0808 22:56:35.900327 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:35.900332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:35.900344 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:56:35.900747 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:35.900751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:35.900767 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:35.900863 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:56:35.901987 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.903058 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.904994 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.906532 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.906608 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.906685 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:35.907727 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0808 22:56:35.907782 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1315,"thread_start_us":206,"threads_started":1}
I20260808 22:56:35.908427 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:35.898364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43193 (local address 127.0.0.1:33502)
0808 22:56:35.899516 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:35.899567 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:35.900621 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:35.901845 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:35.901846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:35.901849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:56:35.901902 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.901903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.903740 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.903741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.904872 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:35.904874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:35.906725 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:35.906726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:35.906796 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:35.906800 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:56:35.906853 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:56:35.906898 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260808 22:56:35.911312 14539 tablet_service.cc:1467] Processing CreateTablet for tablet bf14e8c439764c86a8363922cc3baef0 (DEFAULT_TABLE table=test-table [id=37713b65e9bc44fdbb32c9ee86e7c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:56:35.911715 14539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf14e8c439764c86a8363922cc3baef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:56:35.910742 14634 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Bootstrap starting.
I20260808 22:56:35.912954 14635 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Bootstrap starting.
I20260808 22:56:35.913924 14635 tablet_bootstrap.cc:654] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Neither blocks nor log segments found. Creating new log.
I20260808 22:56:35.914345 14634 tablet_bootstrap.cc:654] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Neither blocks nor log segments found. Creating new log.
I20260808 22:56:35.914713 14634 log.cc:824] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:56:35.915087 14635 log.cc:824] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Log is configured to *not* fsync() on all Append() calls
I20260808 22:56:35.916064 14635 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: No bootstrap required, opened a new log
I20260808 22:56:35.916229 14635 ts_tablet_manager.cc:1397] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:56:35.916242 14634 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: No bootstrap required, opened a new log
I20260808 22:56:35.916308 14634 ts_tablet_manager.cc:1397] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 22:56:35.918582 14634 raft_consensus.cc:348] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.918850 14634 raft_consensus.cc:374] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:56:35.918896 14634 raft_consensus.cc:729] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825d189785f14a44926ba2b66e34577b, State: Initialized, Role: FOLLOWER
I20260808 22:56:35.919211 14637 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Bootstrap starting.
I20260808 22:56:35.919925 14635 raft_consensus.cc:348] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.920199 14635 raft_consensus.cc:374] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:56:35.920301 14635 raft_consensus.cc:729] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a78e5ef6eb90400db00a4207d0460556, State: Initialized, Role: FOLLOWER
I20260808 22:56:35.920512 14637 tablet_bootstrap.cc:654] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Neither blocks nor log segments found. Creating new log.
I20260808 22:56:35.920522 14635 consensus_queue.cc:260] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.920876 14637 log.cc:824] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Log is configured to *not* fsync() on all Append() calls
I20260808 22:56:35.921108 14635 ts_tablet_manager.cc:1428] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:56:35.921187 14634 consensus_queue.cc:260] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.921301 14320 heartbeater.cc:503] Master 127.1.212.62:39289 was elected leader, sending a full tablet report...
I20260808 22:56:35.921478 14634 ts_tablet_manager.cc:1428] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:56:35.921597 14466 heartbeater.cc:503] Master 127.1.212.62:39289 was elected leader, sending a full tablet report...
I20260808 22:56:35.923566 14637 tablet_bootstrap.cc:492] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: No bootstrap required, opened a new log
I20260808 22:56:35.923655 14637 ts_tablet_manager.cc:1397] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:56:35.925894 14637 raft_consensus.cc:348] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.926106 14637 raft_consensus.cc:374] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:56:35.926136 14637 raft_consensus.cc:729] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ec2f457fa4456684c5dd31bcfe5afe, State: Initialized, Role: FOLLOWER
I20260808 22:56:35.926290 14637 consensus_queue.cc:260] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:35.926658 14637 ts_tablet_manager.cc:1428] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:56:35.926779 14614 heartbeater.cc:503] Master 127.1.212.62:39289 was elected leader, sending a full tablet report...
Aug 08 22:56:35 dist-test-slave-1x4g krb5kdc[13592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229795, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
W20260808 22:56:36.048707 14322 tablet.cc:2369] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:56:36.078193 14467 tablet.cc:2369] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:56:36.080322 14639 raft_consensus.cc:482] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:56:36.080480 14639 raft_consensus.cc:504] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:36.080963 14639 leader_election.cc:288] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28ec2f457fa4456684c5dd31bcfe5afe (127.1.212.3:43193), 825d189785f14a44926ba2b66e34577b (127.1.212.2:35521)
I20260808 22:56:36.084967 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.081224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:35521 (local address 127.1.212.1:42363)
0808 22:56:36.081606 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:36.081618 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:36.081728 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:36.081925 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:36.081926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:36.081928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:56:36.082035 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:36.082037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.083026 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.083027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:36.083870 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:36.083873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.084638 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.084639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:36.084726 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:36.084731 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:56:36.084820 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:56:36.084864 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260808 22:56:36.084972 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.081440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:42363 (local address 127.1.212.2:35521)
0808 22:56:36.081546 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:36.081548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:36.081686 (+ 138us) server_negotiation.cc:407] Connection header received
0808 22:56:36.081811 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:36.081812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:36.081822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:36.081853 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:56:36.082132 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.082886 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.083980 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.084529 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.084624 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:36.084637 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:36.084850 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 22:56:36.084895 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 22:56:36.085312 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf14e8c439764c86a8363922cc3baef0" candidate_uuid: "a78e5ef6eb90400db00a4207d0460556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825d189785f14a44926ba2b66e34577b" is_pre_election: true
I20260808 22:56:36.085520 14419 raft_consensus.cc:2454] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a78e5ef6eb90400db00a4207d0460556 in term 0.
I20260808 22:56:36.085770 14200 leader_election.cc:302] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825d189785f14a44926ba2b66e34577b, a78e5ef6eb90400db00a4207d0460556; no voters:
I20260808 22:56:36.085947 14639 raft_consensus.cc:2781] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:56:36.086009 14639 raft_consensus.cc:482] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:56:36.086031 14639 raft_consensus.cc:3037] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:56:36.086298 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.081403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:42545 (local address 127.1.212.3:43193)
0808 22:56:36.081505 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:36.081507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:36.081658 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:56:36.081831 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:36.081833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:36.081842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:36.081874 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:56:36.082160 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.082899 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.085382 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.085897 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.085972 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:36.085983 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:36.086189 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 22:56:36.086230 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:56:36.086527 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.081177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43193 (local address 127.1.212.1:42545)
0808 22:56:36.081551 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:36.081564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:56:36.081724 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:56:36.081945 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:56:36.081946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:56:36.081948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:56:36.082064 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:36.082067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.084135 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.084135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:36.085267 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:56:36.085270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.086021 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.086022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:56:36.086104 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:36.086108 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:56:36.086400 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 22:56:36.086450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260808 22:56:36.086927 14563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf14e8c439764c86a8363922cc3baef0" candidate_uuid: "a78e5ef6eb90400db00a4207d0460556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" is_pre_election: true
I20260808 22:56:36.087111 14563 raft_consensus.cc:2454] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a78e5ef6eb90400db00a4207d0460556 in term 0.
I20260808 22:56:36.087157 14639 raft_consensus.cc:504] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:36.087299 14639 leader_election.cc:288] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [CANDIDATE]: Term 1 election: Requested vote from peers 28ec2f457fa4456684c5dd31bcfe5afe (127.1.212.3:43193), 825d189785f14a44926ba2b66e34577b (127.1.212.2:35521)
I20260808 22:56:36.087482 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf14e8c439764c86a8363922cc3baef0" candidate_uuid: "a78e5ef6eb90400db00a4207d0460556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825d189785f14a44926ba2b66e34577b"
I20260808 22:56:36.087567 14419 raft_consensus.cc:3037] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:56:36.087584 14563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf14e8c439764c86a8363922cc3baef0" candidate_uuid: "a78e5ef6eb90400db00a4207d0460556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ec2f457fa4456684c5dd31bcfe5afe"
I20260808 22:56:36.087668 14563 raft_consensus.cc:3037] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:56:36.088250 14419 raft_consensus.cc:2454] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a78e5ef6eb90400db00a4207d0460556 in term 1.
I20260808 22:56:36.088419 14200 leader_election.cc:302] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825d189785f14a44926ba2b66e34577b, a78e5ef6eb90400db00a4207d0460556; no voters:
I20260808 22:56:36.088546 14639 raft_consensus.cc:2781] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:56:36.088559 14563 raft_consensus.cc:2454] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a78e5ef6eb90400db00a4207d0460556 in term 1.
I20260808 22:56:36.088765 14639 raft_consensus.cc:686] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [term 1 LEADER]: Becoming Leader. State: Replica: a78e5ef6eb90400db00a4207d0460556, State: Running, Role: LEADER
I20260808 22:56:36.088912 14639 consensus_queue.cc:237] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } }
I20260808 22:56:36.089790 14092 catalog_manager.cc:5495] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 reported cstate change: term changed from 0 to 1, leader changed from <none> to a78e5ef6eb90400db00a4207d0460556 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "a78e5ef6eb90400db00a4207d0460556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a78e5ef6eb90400db00a4207d0460556" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 44011 } health_report { overall_health: HEALTHY } } }
W20260808 22:56:36.106424 14615 tablet.cc:2369] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:56:36.135176 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.132347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.1.212.1:44011)
0808 22:56:36.132410 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:56:36.132412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:56:36.132421 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:56:36.132471 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:56:36.132472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:56:36.132500 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:56:36.132622 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:56:36.132825 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.133319 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.134123 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:56:36.134394 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:56:36.134473 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:56:36.134650 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:56:36.134778 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:56:36.134872 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:56:36.134927 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:56:36.134953 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:56:36.137166 14563 raft_consensus.cc:1261] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 1 FOLLOWER]: Refusing update from remote peer a78e5ef6eb90400db00a4207d0460556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:56:36.137138 14419 raft_consensus.cc:1261] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 1 FOLLOWER]: Refusing update from remote peer a78e5ef6eb90400db00a4207d0460556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:56:36.137523 14639 consensus_queue.cc:1035] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ec2f457fa4456684c5dd31bcfe5afe" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:56:36.137655 14645 consensus_queue.cc:1035] T bf14e8c439764c86a8363922cc3baef0 P a78e5ef6eb90400db00a4207d0460556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825d189785f14a44926ba2b66e34577b" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 35521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:56:36.140765 14649 mvcc.cc:205] Tried to move back new op lower bound from 7316397244974804992 to 7316397244780859392. Current Snapshot: MvccSnapshot[applied={T|T < 7316397244974804992}]
I20260808 22:56:36.142719 14652 mvcc.cc:205] Tried to move back new op lower bound from 7316397244974804992 to 7316397244780859392. Current Snapshot: MvccSnapshot[applied={T|T < 7316397244974804992}]
I20260808 22:56:36.143991 14650 mvcc.cc:205] Tried to move back new op lower bound from 7316397244974804992 to 7316397244780859392. Current Snapshot: MvccSnapshot[applied={T|T < 7316397244974804992}]
I20260808 22:56:36.145718 14092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:56:36.145843 14092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:56:36.146633 14092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 124e5d9e9c6f4dabaa700f1ca4627be1: Sending DeleteTablet for 3 replicas of tablet bf14e8c439764c86a8363922cc3baef0
I20260808 22:56:36.147199 14539 tablet_service.cc:1514] Processing DeleteTablet for tablet bf14e8c439764c86a8363922cc3baef0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:56:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:33502
I20260808 22:56:36.147476 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet bf14e8c439764c86a8363922cc3baef0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:56:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:45450
I20260808 22:56:36.147902 14248 tablet_service.cc:1514] Processing DeleteTablet for tablet bf14e8c439764c86a8363922cc3baef0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:56:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:44382
I20260808 22:56:36.148437 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 14180
I20260808 22:56:36.151144 14661 tablet_replica.cc:330] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: stopping tablet replica
I20260808 22:56:36.151316 14661 raft_consensus.cc:2229] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:56:36.151482 14661 raft_consensus.cc:2258] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:56:36.151708 14660 tablet_replica.cc:330] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: stopping tablet replica
I20260808 22:56:36.151829 14660 raft_consensus.cc:2229] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:56:36.151970 14660 raft_consensus.cc:2258] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:56:36.152247 14661 ts_tablet_manager.cc:1905] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:56:36.154785 14661 ts_tablet_manager.cc:1918] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:56:36.154959 14661 log.cc:1196] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/bf14e8c439764c86a8363922cc3baef0
I20260808 22:56:36.155359 14661 ts_tablet_manager.cc:1939] T bf14e8c439764c86a8363922cc3baef0 P 825d189785f14a44926ba2b66e34577b: Deleting consensus metadata
I20260808 22:56:36.155946 14079 catalog_manager.cc:4841] TS 825d189785f14a44926ba2b66e34577b (127.1.212.2:35521): tablet bf14e8c439764c86a8363922cc3baef0 (table test-table [id=37713b65e9bc44fdbb32c9ee86e7c285]) successfully deleted
I20260808 22:56:36.157675 14660 ts_tablet_manager.cc:1905] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:56:36.159163 14077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:44011 (error 108)
I20260808 22:56:36.159415 14660 ts_tablet_manager.cc:1918] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:56:36.159497 14660 log.cc:1196] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/bf14e8c439764c86a8363922cc3baef0
I20260808 22:56:36.160013 14660 ts_tablet_manager.cc:1939] T bf14e8c439764c86a8363922cc3baef0 P 28ec2f457fa4456684c5dd31bcfe5afe: Deleting consensus metadata
I20260808 22:56:36.160529 14077 catalog_manager.cc:4841] TS 28ec2f457fa4456684c5dd31bcfe5afe (127.1.212.3:43193): tablet bf14e8c439764c86a8363922cc3baef0 (table test-table [id=37713b65e9bc44fdbb32c9ee86e7c285]) successfully deleted
I20260808 22:56:36.160575 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 14327
I20260808 22:56:36.165638 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:56:36.165143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:44011 (local address 127.0.0.1:44410)
0808 22:56:36.165550 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 22:56:36.165604 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
W20260808 22:56:36.165835 14077 catalog_manager.cc:4584] TS a78e5ef6eb90400db00a4207d0460556 (127.1.212.1:44011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf14e8c439764c86a8363922cc3baef0: Network error: Client connection negotiation failed: client connection to 127.1.212.1:44011: connect: Connection refused (error 111)
I20260808 22:56:36.168120 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 14472
I20260808 22:56:36.175347 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 14058
I20260808 22:56:36.183780 1872 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:56:36.528589 1872 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:56:36.528687 1872 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:56:36.873998 1872 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:56:36.874 UTC [13622] LOG: received smart shutdown request
2026-08-08 22:56:36.878 UTC [13622] LOG: background worker "logical replication launcher" (PID 13632) exited with exit code 1
2026-08-08 22:56:36.879 UTC [13626] LOG: shutting down
2026-08-08 22:56:37.070 UTC [13622] LOG: database system is shut down
2026-08-08T22:56:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50639 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:56:37 dist-test-slave-1x4g krb5kdc[14672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:56:37 dist-test-slave-1x4g krb5kdc[14672](info): set up 2 sockets
Aug 08 22:56:37 dist-test-slave-1x4g krb5kdc[14672](info): commencing operation
krb5kdc: starting...
W20260808 22:56:38.265985 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:56:38 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:56:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangeradmin_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangeradmin_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangerlookup_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangerlookup_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
I20260808 22:56:38.465046 1872 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskusC_hJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/postgres -l logfile start
2026-08-08 22:56:41.120 UTC [14706] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:56:41.120 UTC [14706] LOG: listening on IPv4 address "127.1.212.20", port 34337
2026-08-08 22:56:41.124 UTC [14706] LOG: listening on Unix socket "/tmp/.s.PGSQL.34337"
2026-08-08 22:56:41.147 UTC [14709] LOG: database system was shut down at 2026-08-08 22:56:39 UTC
2026-08-08 22:56:41.151 UTC [14706] LOG: database system is ready to accept connections
I20260808 22:56:42.133175 1872 mini_postgres.cc:96] Postgres bound to 34337
2026-08-08 22:56:42.140 UTC [14718] FATAL: database "slave" does not exist
127.1.212.20:34337 - accepting connections
I20260808 22:56:42.141173 1872 mini_ranger.cc:161] Starting Ranger...
I20260808 22:56:42.158170 1872 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:56:42.678218 1872 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:56:42.678298 1872 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin
2026-08-08 22:56:42,961 [I] DB FLAVOR :POSTGRES
2026-08-08 22:56:42,962 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:56:42,962 [I] Checking connection..
2026-08-08 22:56:42,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:56:43,224 [I] Checking connection passed.
2026-08-08 22:56:43,225 [I] --------- Verifying version history table ---------
2026-08-08 22:56:43,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:56:43,497 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:56:43,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:56:43,751 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:56:43,752 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:56:43,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:56:43.999 UTC [14790] WARNING: there is no transaction in progress
2026-08-08 22:56:44,008 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:56:44,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:56:44,285 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:56:44,285 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:56:44,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:56:44,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:56:44,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-08 22:56:45,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:56:45,310 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:56:45,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:56:45,562 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:56:45,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:56:45,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:56:46,072 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:56:46,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:56:47.173 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.191 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.209 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.226 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.247 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.271 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.344 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.353 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.372 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.383 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.395 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.412 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.424 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.432 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.441 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:47.449 UTC [14904] WARNING: there is no transaction in progress
2026-08-08 22:56:48,206 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:56:48,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:56:48,452 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:56:48,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:56:48,721 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:56:48,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:56:49,002 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:56:49,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:56:49,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:56:49,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-08 22:56:49,771 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:56:49,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:56:50,019 [I] DB_PATCHES have already been applied
I20260808 22:56:50.027329 1872 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskusC_hJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:56:50.027434 1872 mini_ranger.cc:193] Using host: 127.1.212.20
I20260808 22:56:50.033102 1872 mini_ranger.cc:227] Ranger admin URL: http://127.1.212.20:46711
Aug 08, 2026 10:56:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:56:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:56:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:56:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:56:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:56:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:56:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:56:51.218109 1872 mini_ranger.cc:160] Time spent starting Ranger: real 9.077s user 0.002s sys 0.010s
Aug 08, 2026 10:57:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:57:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 10:57:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:57:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:57:15 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:57:15.622653 1872 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangerkms_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/rangerkms_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.212.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.212.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.212.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/HTTP_127.1.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:57:15 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229835, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:57:15.718586 1872 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:57:15.730226 1872 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:57:16.719120 1872 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:57:16.719228 1872 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms
I20260808 22:57:16.719251 1872 mini_ranger_kms.cc:227] Using postgres at 127.1.212.20:34337
2026-08-08 22:57:16,921 [I] DB FLAVOR :POSTGRES
2026-08-08 22:57:16,921 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:57:16,921 [I] Checking connection
2026-08-08 22:57:16,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:57:17,178 [I] connection success
2026-08-08 22:57:17,178 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:57:17,179 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:57:17,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:57:17,427 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:57:17,428 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:57:17,428 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:57:17,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.212.20:34337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:57:17,732 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:57:18.302898 1872 mini_ranger_kms.cc:310] Created kms service
I20260808 22:57:18.529479 1872 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:57:18.577153 1872 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:57:18 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229835, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:19.039431 1872 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:57:19.039661 1872 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms:/tmp/dist-test-taskusC_hJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskusC_hJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:57:19.039680 1872 mini_ranger_kms.cc:240] Using host: 127.1.212.20
I20260808 22:57:19.044143 1872 mini_ranger_kms.cc:276] Ranger KMS PID: 15096
I20260808 22:57:19.044243 1872 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.212.20:33489
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:57:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 22:57:21.505131 1872 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.787s user 0.009s sys 0.173s
I20260808 22:57:21.505232 1872 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:57:21.505280 1872 mini_ranger_kms.cc:401] 127.1.212.20:33489/kms/v1/keys
Aug 08 22:57:21 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786229841, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 22:57:22 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229841, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
Aug 08 22:57:22 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229841, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.212.20@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
Aug 08 22:57:24 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:57:25.016888 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43565
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:43565
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:25.106595 15147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:25.106855 15147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:25.106881 15147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:25.108474 15147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:25.108557 15147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:25.108572 15147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:25.108587 15147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:25.108599 15147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:25.110909 15147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.212.20:33489/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:43565
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:25.111266 15147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:25.111549 15147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:25.117247 15154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:25.117316 15147 server_base.cc:1034] running on GCE node
W20260808 22:57:25.117247 15153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:25.117468 15156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:25.118002 15147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:25.118355 15147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:25.119544 15147 hybrid_clock.cc:648] HybridClock initialized: now 1786229845119509 us; error 54 us; skew 500 ppm
Aug 08 22:57:25 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229845, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:25.122515 15147 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:25.123170 15147 webserver.cc:460] Webserver started at http://127.1.212.62:37077/ using document root <none> and password file <none>
I20260808 22:57:25.123430 15147 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:25.123531 15147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:25.123720 15147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:57:25 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229845, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:25.187978 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1"
format_stamp: "Formatted at 2026-08-08 22:57:25 on dist-test-slave-1x4g"
server_key: "5e5186c6848a96a25bc1587f2e09dfe7"
server_key_iv: "d076a728d2ea767434b70f02d7f8be32"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:25.188513 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1"
format_stamp: "Formatted at 2026-08-08 22:57:25 on dist-test-slave-1x4g"
server_key: "5e5186c6848a96a25bc1587f2e09dfe7"
server_key_iv: "d076a728d2ea767434b70f02d7f8be32"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:25.190189 15147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 22:57:25.207377 15163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:25.207624 15147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:25.207726 15147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1"
format_stamp: "Formatted at 2026-08-08 22:57:25 on dist-test-slave-1x4g"
server_key: "5e5186c6848a96a25bc1587f2e09dfe7"
server_key_iv: "d076a728d2ea767434b70f02d7f8be32"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:25.207823 15147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:25.228873 15147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:25.231416 15147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:25.231590 15147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:25.237000 15147 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:43565
I20260808 22:57:25.237082 15219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:43565 every 8 connection(s)
I20260808 22:57:25.237483 15147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:25.238231 15220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:25.240655 15220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Bootstrap starting.
I20260808 22:57:25.241346 15220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:25.241667 15220 log.cc:824] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:25.242420 15220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: No bootstrap required, opened a new log
I20260808 22:57:25.244581 15220 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:25.244830 15220 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:25.244887 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15147
I20260808 22:57:25.244901 15220 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e3a1a22cd64d81a2c45f6d4d8d6ea1, State: Initialized, Role: FOLLOWER
I20260808 22:57:25.244997 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:57:25.245113 15220 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:25.245158 1872 external_mini_cluster.cc:1428] Setting key 747bacecaea0bc8871eb72550423f5cd
I20260808 22:57:25.245215 15220 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:25.245275 15220 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:25.245337 15220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:25.246186 15220 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:25.246322 15220 leader_election.cc:302] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e3a1a22cd64d81a2c45f6d4d8d6ea1; no voters:
I20260808 22:57:25.246588 15220 leader_election.cc:288] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:57:25.246652 15225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:25.246865 15225 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 LEADER]: Becoming Leader. State: Replica: c7e3a1a22cd64d81a2c45f6d4d8d6ea1, State: Running, Role: LEADER
I20260808 22:57:25.246963 15220 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:25.247026 15225 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:25.247431 15225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } } }
I20260808 22:57:25.247627 15225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:25.247813 15225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e3a1a22cd64d81a2c45f6d4d8d6ea1. Latest consensus state: current_term: 1 leader_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } } }
I20260808 22:57:25.247912 15225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:25.248075 15234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:25.248620 15234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:25.248968 15220 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:57:25.255877 15234 catalog_manager.cc:1347] Generated new cluster ID: 379c2f685c2a4e838794d8f6416ea48e
Aug 08 22:57:25 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:25.256016 15234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:25.261637 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:25.245691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.1.212.62:43565)
0808 22:57:25.245910 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:25.245921 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:25.245955 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:57:25.246093 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:25.246101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:25.246158 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:25.246307 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:25.247273 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:25.248848 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:25.251425 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:25.253137 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:25.253326 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:25.256670 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:25.256676 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:25.256677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:25.256692 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:25.259297 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:25.259758 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:25.259762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:25.259763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:25.259920 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:25.260150 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:25.260150 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:25.260151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:25.260484 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:25.260624 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:25.261218 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 22:57:25.261361 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260808 22:57:25.262861 15234 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:57:25.263682 15234 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:25.274456 15234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Generated new TSK 0
I20260808 22:57:25.274778 15234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:57:26.075654 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:26.174777 15266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:26.175014 15266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:26.175041 15266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:26.176421 15266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:26.176489 15266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:26.176520 15266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:26.178363 15266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:26.178653 15266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:26.178952 15266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:26.185449 15272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.185449 15271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.185978 15274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:26.186291 15266 server_base.cc:1034] running on GCE node
I20260808 22:57:26.186534 15266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:26.186823 15266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:26.188043 15266 hybrid_clock.cc:648] HybridClock initialized: now 1786229846188021 us; error 35 us; skew 500 ppm
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:26.191033 15266 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:26.191560 15266 webserver.cc:460] Webserver started at http://127.1.212.1:41277/ using document root <none> and password file <none>
I20260808 22:57:26.191830 15266 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:26.191901 15266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:26.192060 15266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:26.212440 15266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "528440a714a003db8aaa7280c9ba3ad8"
server_key_iv: "ebf08d808f72e392983008b090384bb1"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.213254 15266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "528440a714a003db8aaa7280c9ba3ad8"
server_key_iv: "ebf08d808f72e392983008b090384bb1"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.216197 15266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 22:57:26.230799 15281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.231072 15266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:26.231173 15266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "528440a714a003db8aaa7280c9ba3ad8"
server_key_iv: "ebf08d808f72e392983008b090384bb1"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.231307 15266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:26.262012 15266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:26.265094 15266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:26.265318 15266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:26.265610 15266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:26.266163 15266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:26.266232 15266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.266292 15266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:26.266340 15266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.273905 15266 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:36531
I20260808 22:57:26.274538 15266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:26.275983 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15266
I20260808 22:57:26.276100 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:57:26.276332 1872 external_mini_cluster.cc:1428] Setting key 78ae6a8d3e8a29f1a08058aae39010f2
I20260808 22:57:26.278093 15404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:36531 every 8 connection(s)
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:26.291412 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.278783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:51913 (local address 127.1.212.62:43565)
0808 22:57:26.278898 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:26.278901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:26.279956 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 22:57:26.279992 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:26.279993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:26.280010 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:26.280076 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:26.281040 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.281867 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.283580 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.283928 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.284011 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.287087 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:26.287090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:26.287090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:26.287101 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:26.289198 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.289912 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.289914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.289914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.290004 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.290365 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.290366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.290367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.290555 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:26.290614 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:26.291110 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 22:57:26.291241 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260808 22:57:26.291563 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.278755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.1:51913)
0808 22:57:26.279054 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:26.279103 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:26.279787 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:26.280218 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:26.280224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:26.280545 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:26.280912 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.280920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.282013 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.282014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.283438 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.283458 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.284071 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.284073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.284242 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.285067 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:26.285079 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:26.286943 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:26.289333 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.289347 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.289351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.289788 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.290098 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.290098 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.290099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.290279 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.290673 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:26.290683 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:26.290912 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:57:26.291053 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:57:26.291267 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260808 22:57:26.292879 15405 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:26.293044 15405 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:26.293296 15405 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:26.294072 15181 ts_manager.cc:194] Registered new tserver with Master: 84c3a9ad1fb5419297dcc0b18f0a4b7f (127.1.212.1:36531)
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
I20260808 22:57:26.295094 15181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:51913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:57:26.327834 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 22:57:26.458702 15412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:26.459093 15412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:26.459192 15412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:26.461229 15412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:26.461361 15412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:26.461452 15412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:26.464051 15412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:26.464614 15412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:26.465016 15412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:26.472204 15420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.472653 15419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.472956 15423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:26.474167 15412 server_base.cc:1034] running on GCE node
I20260808 22:57:26.474419 15412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:26.474727 15412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:26.475939 15412 hybrid_clock.cc:648] HybridClock initialized: now 1786229846475916 us; error 37 us; skew 500 ppm
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:26.479702 15412 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:26.480316 15412 webserver.cc:460] Webserver started at http://127.1.212.2:37359/ using document root <none> and password file <none>
I20260808 22:57:26.480607 15412 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:26.480664 15412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:26.480839 15412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:26.505331 15412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "e2cd62cf058147c2ae6c14be2e6d8c89"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "50544a3d872e8c29eb277e1cf2453f0b"
server_key_iv: "aa10c9c4412cc60eee5ec78d10239b1e"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.505828 15412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "e2cd62cf058147c2ae6c14be2e6d8c89"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "50544a3d872e8c29eb277e1cf2453f0b"
server_key_iv: "aa10c9c4412cc60eee5ec78d10239b1e"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.509389 15412 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:57:26.523994 15430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.524281 15412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:26.524394 15412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "e2cd62cf058147c2ae6c14be2e6d8c89"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "50544a3d872e8c29eb277e1cf2453f0b"
server_key_iv: "aa10c9c4412cc60eee5ec78d10239b1e"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.524501 15412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:26.556371 15412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:26.559322 15412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:26.559581 15412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:26.559947 15412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:26.560456 15412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:26.560546 15412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.560623 15412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:26.560683 15412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.568228 15412 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:40975
I20260808 22:57:26.569109 15412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:57:26.571991 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15412
I20260808 22:57:26.572110 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:57:26.572311 1872 external_mini_cluster.cc:1428] Setting key 7a7e6017ad04a603c10d5436d86f1521
I20260808 22:57:26.573447 15551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:40975 every 8 connection(s)
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:26.588462 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.574535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:35667 (local address 127.1.212.62:43565)
0808 22:57:26.574587 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:26.574590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:26.576108 (+ 1518us) server_negotiation.cc:407] Connection header received
0808 22:57:26.576968 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:26.576970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:26.576985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:26.577020 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:26.578075 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.578852 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.580425 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.580818 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.580876 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.584466 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:26.584469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:26.584469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:26.584483 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:26.586413 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.587051 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.587053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.587053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.587125 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.587485 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.587486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.587487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.587652 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:26.587727 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:26.588243 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 22:57:26.588360 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:57:26.588665 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.574204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.2:35667)
0808 22:57:26.575904 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:26.575967 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:26.576825 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:26.577259 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:26.577264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:26.577601 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:26.577930 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.577938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.578989 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.578991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.580288 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.580292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.580999 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.581000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.581149 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.582372 (+ 1223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:26.582385 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:26.584320 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:26.586540 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.586552 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.586556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.586943 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.587219 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.587220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.587220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.587397 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.587745 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:26.587754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:26.588050 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 22:57:26.588192 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 22:57:26.588394 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1585,"thread_start_us":118,"threads_started":1}
I20260808 22:57:26.589500 15552 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:26.589635 15552 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:26.589867 15552 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:26.590394 15181 ts_manager.cc:194] Registered new tserver with Master: e2cd62cf058147c2ae6c14be2e6d8c89 (127.1.212.2:40975)
I20260808 22:57:26.590925 15181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:35667
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:57:26.618038 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:26.758457 15557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:26.758798 15557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:26.758838 15557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:26.760854 15557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:26.760933 15557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:26.760980 15557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:26.763481 15557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:26.763808 15557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:26.764168 15557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:26.770939 15563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.771080 15565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:26.771330 15562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:26.771667 15557 server_base.cc:1034] running on GCE node
I20260808 22:57:26.771893 15557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:26.772185 15557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:26.773360 15557 hybrid_clock.cc:648] HybridClock initialized: now 1786229846773339 us; error 39 us; skew 500 ppm
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:26.777274 15557 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:26.778024 15557 webserver.cc:460] Webserver started at http://127.1.212.3:34889/ using document root <none> and password file <none>
I20260808 22:57:26.778419 15557 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:26.778502 15557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:26.778687 15557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:26.804256 15557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "7fd59636f65d4d9681388d499d6abc5f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "2c29ad5091d828d48e48eb53acdc1233"
server_key_iv: "ea110cb6fb2fd4b804a8effd469c8656"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.804888 15557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "7fd59636f65d4d9681388d499d6abc5f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "2c29ad5091d828d48e48eb53acdc1233"
server_key_iv: "ea110cb6fb2fd4b804a8effd469c8656"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.813306 15557 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260808 22:57:26.841554 15572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.841863 15557 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.001s sys 0.000s
I20260808 22:57:26.841970 15557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "7fd59636f65d4d9681388d499d6abc5f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "2c29ad5091d828d48e48eb53acdc1233"
server_key_iv: "ea110cb6fb2fd4b804a8effd469c8656"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:26.842078 15557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:26.875516 15557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:26.890730 15557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:26.890923 15557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:26.891175 15557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:26.891630 15557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:26.891687 15557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.891721 15557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:26.891749 15557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:26.899495 15557 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:37879
I20260808 22:57:26.900080 15557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:57:26.902098 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15557
I20260808 22:57:26.902210 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:57:26.902429 1872 external_mini_cluster.cc:1428] Setting key 0603877abbf202fea462c17986f63819
I20260808 22:57:26.911442 15692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:37879 every 8 connection(s)
Aug 08 22:57:26 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229846, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:26.924345 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.913399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:43629 (local address 127.1.212.62:43565)
0808 22:57:26.913473 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:26.913476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:26.913855 (+ 379us) server_negotiation.cc:407] Connection header received
0808 22:57:26.914564 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:26.914565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:26.914578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:26.914608 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:26.915511 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.916222 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.917645 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.917887 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.917942 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.920610 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:26.920612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:26.920612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:26.920621 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:26.922577 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.923180 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.923181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.923182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.923247 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:26.923536 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:26.923537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:26.923537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:26.923678 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:26.923720 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:26.924153 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 22:57:26.924245 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:57:26.924944 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:26.913413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.3:43629)
0808 22:57:26.913725 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:26.913769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:26.914440 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:26.914803 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:26.914808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:26.915094 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:26.915401 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.915409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.916354 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.916355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.917536 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:26.917541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:26.918060 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:26.918061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:26.918208 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:26.918844 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:26.918857 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:26.920480 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:26.922704 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.922720 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.922724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.923083 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.923317 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:26.923318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:26.923318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:26.923462 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:26.923766 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:26.923774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:26.923991 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 22:57:26.924475 (+ 484us) client_negotiation.cc:240] Negotiation successful
0808 22:57:26.924667 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260808 22:57:26.925379 15693 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:26.925496 15693 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:26.925720 15693 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:26.926154 15181 ts_manager.cc:194] Registered new tserver with Master: 7fd59636f65d4d9681388d499d6abc5f (127.1.212.3:37879)
I20260808 22:57:26.926633 15181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:43629
I20260808 22:57:26.936364 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:57:26.937158 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15266
I20260808 22:57:26.946688 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15412
I20260808 22:57:26.954875 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15557
I20260808 22:57:26.964560 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15147
I20260808 22:57:26.973022 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43565
--webserver_interface=127.1.212.62
--webserver_port=37077
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:43565
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:27.061515 15696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:27.061761 15696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:27.061785 15696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:27.063051 15696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:27.063105 15696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:27.063120 15696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:27.063133 15696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:27.063145 15696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:27.064991 15696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:43565
--ranger_config_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=37077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:27.065290 15696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:27.065562 15696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:27.070894 15705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.070896 15701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.070904 15702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:27.071189 15696 server_base.cc:1034] running on GCE node
I20260808 22:57:27.071475 15696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:27.071869 15696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:27.073020 15696 hybrid_clock.cc:648] HybridClock initialized: now 1786229847072993 us; error 42 us; skew 500 ppm
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:27.075830 15696 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:27.076395 15696 webserver.cc:460] Webserver started at http://127.1.212.62:37077/ using document root <none> and password file <none>
I20260808 22:57:27.076671 15696 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:27.076757 15696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:27.121004 15696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:27.121824 15713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.122027 15696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:27.122148 15696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1"
format_stamp: "Formatted at 2026-08-08 22:57:25 on dist-test-slave-1x4g"
server_key: "5e5186c6848a96a25bc1587f2e09dfe7"
server_key_iv: "d076a728d2ea767434b70f02d7f8be32"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:27.122550 15696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:27.139596 15696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:27.141937 15696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:27.142125 15696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:27.146776 15696 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:43565
I20260808 22:57:27.146826 15769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:43565 every 8 connection(s)
I20260808 22:57:27.147235 15696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:27.148568 15770 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:57:27.149374 15770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Bootstrap starting.
I20260808 22:57:27.151459 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15696
I20260808 22:57:27.151810 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:36531
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:43565
--webserver_port=41277
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:27.158958 15770 log.cc:824] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:27.161044 15770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:57:27.161404 15770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Bootstrap complete.
I20260808 22:57:27.164067 15770 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:27.164335 15770 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e3a1a22cd64d81a2c45f6d4d8d6ea1, State: Initialized, Role: FOLLOWER
I20260808 22:57:27.164525 15770 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:27.164611 15770 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:27.164680 15770 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:27.164721 15770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:57:27.165949 15770 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:27.166088 15770 leader_election.cc:302] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e3a1a22cd64d81a2c45f6d4d8d6ea1; no voters:
I20260808 22:57:27.166311 15770 leader_election.cc:288] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:57:27.166355 15775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:57:27.166596 15775 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [term 2 LEADER]: Becoming Leader. State: Replica: c7e3a1a22cd64d81a2c45f6d4d8d6ea1, State: Running, Role: LEADER
I20260808 22:57:27.166656 15770 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:27.166791 15775 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } }
I20260808 22:57:27.167126 15776 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } } }
I20260808 22:57:27.167204 15776 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:27.167148 15777 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e3a1a22cd64d81a2c45f6d4d8d6ea1. Latest consensus state: current_term: 2 leader_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e3a1a22cd64d81a2c45f6d4d8d6ea1" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43565 } } }
I20260808 22:57:27.167258 15777 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:27.167441 15780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:27.168069 15780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:27.168241 15780 catalog_manager.cc:1259] Loaded cluster ID: 379c2f685c2a4e838794d8f6416ea48e
I20260808 22:57:27.168287 15780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:27.168475 15770 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:57:27.172605 15780 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:27.173329 15780 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Loaded TSK: 0
I20260808 22:57:27.173668 15780 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:57:27.264256 15773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:27.264566 15773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:27.264612 15773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:27.266716 15773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:27.266791 15773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:27.266837 15773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:27.269606 15773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:36531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=41277
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:27.269941 15773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:27.270282 15773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:27.277591 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.277751 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.277639 15813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:27.277973 15773 server_base.cc:1034] running on GCE node
I20260808 22:57:27.278419 15773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:27.278736 15773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:27.279927 15773 hybrid_clock.cc:648] HybridClock initialized: now 1786229847279897 us; error 48 us; skew 500 ppm
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:27.283560 15773 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:27.284209 15773 webserver.cc:460] Webserver started at http://127.1.212.1:41277/ using document root <none> and password file <none>
I20260808 22:57:27.284557 15773 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:27.284693 15773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:27.303519 15773 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260808 22:57:27.305433 15821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.305644 15773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:57:27.305748 15773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "528440a714a003db8aaa7280c9ba3ad8"
server_key_iv: "ebf08d808f72e392983008b090384bb1"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:27.306294 15773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:27.326629 15773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:27.329368 15773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:27.329559 15773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:27.329823 15773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:27.330231 15773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:27.330289 15773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.330333 15773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:27.330355 15773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.338318 15773 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:36531
I20260808 22:57:27.338423 15939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:36531 every 8 connection(s)
I20260808 22:57:27.338855 15773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:27.339380 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15773
I20260808 22:57:27.339696 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:40975
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:43565
--webserver_port=37359
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:27.354470 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.340239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:60949 (local address 127.1.212.62:43565)
0808 22:57:27.341272 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:27.341282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:27.341326 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:57:27.341495 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:27.341504 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:27.341572 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:27.342608 (+ 1036us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:27.342714 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.344148 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.345803 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.346493 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.347183 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.349618 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:27.349624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:27.349625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:27.349641 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:27.352006 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.352661 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.352665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.352666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.352849 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.353233 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.353234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.353234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.353604 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:27.353741 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:27.354106 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 22:57:27.354234 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":80,"threads_started":1}
I20260808 22:57:27.354486 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.339819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.1:60949)
0808 22:57:27.340121 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:27.340177 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:27.341021 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:27.341813 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:27.341820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:27.342119 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:27.342445 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.342453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.344292 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.344294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.345664 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.345670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.346654 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.346657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.346824 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.347540 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:27.347553 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:27.349443 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:27.352164 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.352177 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.352180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.352538 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.352951 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.352953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.352953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.353137 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.353722 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:27.353731 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:27.353898 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 22:57:27.354019 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:57:27.354223 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260808 22:57:27.355598 15940 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:27.355720 15940 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.355940 15940 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.357117 15940 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:43565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:57:27.357205 15940 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.357283 15940 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:27.357816 15940 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.357903 15940 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.358317 15940 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:57:27.453503 15944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:27.453732 15944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:27.453759 15944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:27.455096 15944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:27.455150 15944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:27.455180 15944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:27.457436 15944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:40975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=37359
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:27.457733 15944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:27.458032 15944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:57:27.464815 15944 server_base.cc:1034] running on GCE node
W20260808 22:57:27.464905 15951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.465113 15953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.465217 15950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:27.465485 15944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:27.465821 15944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:27.466993 15944 hybrid_clock.cc:648] HybridClock initialized: now 1786229847466980 us; error 32 us; skew 500 ppm
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:27.470402 15944 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:27.471045 15944 webserver.cc:460] Webserver started at http://127.1.212.2:37359/ using document root <none> and password file <none>
I20260808 22:57:27.471362 15944 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:27.471457 15944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:27.497386 15944 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.001s
I20260808 22:57:27.498673 15961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.498863 15944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:27.498963 15944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "e2cd62cf058147c2ae6c14be2e6d8c89"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "50544a3d872e8c29eb277e1cf2453f0b"
server_key_iv: "aa10c9c4412cc60eee5ec78d10239b1e"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:27.499498 15944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:27.522053 15944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:27.524665 15944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:27.524884 15944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:27.525136 15944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:27.525637 15944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:27.525698 15944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.525743 15944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:27.525768 15944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.534047 15944 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:40975
I20260808 22:57:27.534132 16079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:40975 every 8 connection(s)
I20260808 22:57:27.534596 15944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:57:27.541908 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 15944
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:27.542229 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.212.20:33489/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:37879
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:43565
--webserver_port=34889
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:27.547932 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.535485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.2:44083)
0808 22:57:27.535678 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:27.535733 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:27.536252 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:27.536526 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:27.536529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:27.536737 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:27.537025 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.537030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.538038 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.538039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.539085 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.539089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.539664 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.539668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.539860 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.540543 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:27.540554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:27.543627 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:27.545927 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.545939 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.545943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.546323 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.546592 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.546593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.546593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.546756 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.547104 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:27.547112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:27.547315 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 22:57:27.547443 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 22:57:27.547641 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260808 22:57:27.548593 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.535472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:44083 (local address 127.1.212.62:43565)
0808 22:57:27.535525 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:27.535527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:27.535858 (+ 331us) server_negotiation.cc:407] Connection header received
0808 22:57:27.536382 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:27.536383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:27.536398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:27.536468 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:27.537127 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.537928 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.539193 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.539520 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.539602 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.543788 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:27.543791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:27.543791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:27.543801 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:27.545792 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.546439 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.546441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.546441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.546514 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.546838 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.546839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.546839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.547004 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:27.547071 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:27.548387 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0808 22:57:27.548498 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:57:27.549535 16080 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:27.549664 16080 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.549911 16080 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.550608 16080 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:43565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:57:27.550695 16080 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.550777 16080 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:27.551105 16080 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.551196 16080 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.551489 16080 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:57:27.675843 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:27.676201 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:27.676254 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:27.678354 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:27.678442 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:27.678493 16084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:27.681267 16084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.212.20:33489/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:37879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=34889
--tserver_master_addrs=127.1.212.62:43565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:27.681596 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:27.681936 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:27.688184 16089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.688238 16092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:27.688189 16090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:27.688545 16084 server_base.cc:1034] running on GCE node
I20260808 22:57:27.688831 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:27.689174 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:27.690351 16084 hybrid_clock.cc:648] HybridClock initialized: now 1786229847690333 us; error 31 us; skew 500 ppm
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:27.693127 16084 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:27.693635 16084 webserver.cc:460] Webserver started at http://127.1.212.3:34889/ using document root <none> and password file <none>
I20260808 22:57:27.693897 16084 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:27.693987 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:27.713428 16084 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:57:27.714447 16100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.714654 16084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:27.714761 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "7fd59636f65d4d9681388d499d6abc5f"
format_stamp: "Formatted at 2026-08-08 22:57:26 on dist-test-slave-1x4g"
server_key: "2c29ad5091d828d48e48eb53acdc1233"
server_key_iv: "ea110cb6fb2fd4b804a8effd469c8656"
server_key_version: "kuduclusterkey@0"
I20260808 22:57:27.715181 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:27.728938 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:27.731901 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:27.732096 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:27.732429 16084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:27.732856 16084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:27.732898 16084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.732928 16084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:27.732947 16084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:27.740532 16084 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:37879
I20260808 22:57:27.740639 16223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:37879 every 8 connection(s)
I20260808 22:57:27.741077 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:57:27 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:27.751127 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16084
I20260808 22:57:27.751398 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.741954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:43565 (local address 127.1.212.3:41801)
0808 22:57:27.742124 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:27.742163 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:27.742696 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:27.742981 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:27.742985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:27.743181 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:27.743398 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.743403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.744117 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.744120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.745049 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:27.745052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.745486 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.745487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:27.745620 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.746222 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:27.746232 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:27.747603 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:27.749431 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.749441 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.749444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.749786 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.750047 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:27.750047 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:27.750048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:27.750198 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:27.750562 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:27.750569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:27.750759 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 22:57:27.750882 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:57:27.751086 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260808 22:57:27.751924 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.741966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:41801 (local address 127.1.212.62:43565)
0808 22:57:27.742031 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:27.742034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:27.742249 (+ 215us) server_negotiation.cc:407] Connection header received
0808 22:57:27.742809 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:27.742811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:27.742825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:27.742900 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:27.743502 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.744007 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.745149 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.745357 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.745454 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.747714 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:27.747715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:27.747716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:27.747725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:27.749333 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.749896 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.749897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.749898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.749966 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.750276 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.750277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.750277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.750444 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:27.750494 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:27.751726 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0808 22:57:27.751840 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:57:27.753749 16224 heartbeater.cc:347] Connected to a master server at 127.1.212.62:43565
I20260808 22:57:27.753944 16224 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.754220 16224 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.754953 16224 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:43565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:57:27.755034 16224 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.755106 16224 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:27.755559 16224 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:27.755705 16224 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
W20260808 22:57:27.756109 16224 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:57:27.760469 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:27.752904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.1.212.62:43565)
0808 22:57:27.752958 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:27.752961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:27.752970 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:27.752995 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:27.752996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:27.753008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:27.753071 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:27.753931 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.754643 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.755779 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:27.756159 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:27.756228 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:27.757686 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:27.757688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:27.757688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:27.757696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:27.759220 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.759624 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.759625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.759626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.759689 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:27.759919 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:27.759920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:27.759920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:27.760042 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:27.760240 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:27.760278 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 22:57:27.760356 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:57:28.358493 15940 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:28.358673 15940 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:28.359119 15732 ts_manager.cc:194] Registered new tserver with Master: 84c3a9ad1fb5419297dcc0b18f0a4b7f (127.1.212.1:36531)
I20260808 22:57:28.360678 15732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:60949
Aug 08 22:57:28 dist-test-slave-1x4g krb5kdc[14672](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786229848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:28.551684 16080 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:28.551891 16080 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:28.552326 15732 ts_manager.cc:194] Registered new tserver with Master: e2cd62cf058147c2ae6c14be2e6d8c89 (127.1.212.2:40975)
I20260808 22:57:28.552915 15732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:44083
I20260808 22:57:28.756331 16224 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:28.756518 16224 heartbeater.cc:511] Master 127.1.212.62:43565 requested a full tablet report, sending...
I20260808 22:57:28.756949 15732 ts_manager.cc:194] Registered new tserver with Master: 7fd59636f65d4d9681388d499d6abc5f (127.1.212.3:37879)
I20260808 22:57:28.757490 15732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:41801
I20260808 22:57:28.766285 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:57:28.777563 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.767402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.1.212.62:43565)
0808 22:57:28.768917 (+ 1515us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:28.768919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:28.768929 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:57:28.768982 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:28.768982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:28.768998 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:28.769082 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:28.769864 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.770701 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.771786 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.772103 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.772177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.773876 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:28.773878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:28.773879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:28.773889 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:28.775814 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:28.776279 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:28.776280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:28.776280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:28.776348 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:28.776573 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:28.776573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:28.776574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:28.776739 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:28.776878 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:28.777279 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 22:57:28.777376 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":76,"threads_started":1}
I20260808 22:57:28.780264 15732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:57:28.782619 15732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:57:28.792683 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:37879 (local address 127.0.0.1:50550)
0808 22:57:28.787154 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:28.787201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:28.787265 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:28.787637 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:28.787641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:28.787643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:28.787733 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.787735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.789453 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.789454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.790811 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.790815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.792289 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.792290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.792403 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.792424 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:57:28.792521 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:57:28.792594 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"thread_start_us":124,"threads_started":1}
I20260808 22:57:28.792687 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.1.212.3:37879)
0808 22:57:28.787449 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:28.787454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:28.787464 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:57:28.787501 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:28.787503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:28.787519 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:28.787593 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:28.787828 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.789354 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.790933 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.792176 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.792258 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.792324 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:28.792547 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 22:57:28.792595 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":936,"thread_start_us":65,"threads_started":1}
I20260808 22:57:28.794082 16138 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a (DEFAULT_TABLE table=test-table [id=4ce53f08e56845a89e42c676e16d0b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:28.794518 16138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafb13035d24ce0a03d1740ff225a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:28.797799 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:40975 (local address 127.0.0.1:50036)
0808 22:57:28.793409 (+ 6946us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:28.793421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:28.793465 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:28.793722 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:28.793723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:28.793725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:28.793768 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.793769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.795338 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.795340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.796478 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.796482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.797567 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.797568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.797636 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.797640 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:57:28.797670 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 22:57:28.797715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6932,"thread_start_us":74,"threads_started":1}
I20260808 22:57:28.798368 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.1.212.2:40975)
0808 22:57:28.786990 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:28.786995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:28.793542 (+ 6547us) server_negotiation.cc:407] Connection header received
0808 22:57:28.793592 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:28.793596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:28.793616 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:28.793852 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:28.793964 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.795221 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.796613 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.797468 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.798146 (+ 678us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.798219 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:28.798247 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 22:57:28.798290 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260808 22:57:28.799269 16008 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a (DEFAULT_TABLE table=test-table [id=4ce53f08e56845a89e42c676e16d0b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:28.799469 16257 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Bootstrap starting.
I20260808 22:57:28.799639 16008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafb13035d24ce0a03d1740ff225a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:28.800156 16257 tablet_bootstrap.cc:654] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:28.800891 16257 log.cc:824] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:28.800997 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36531 (local address 127.0.0.1:41500)
0808 22:57:28.790880 (+ 4704us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:28.790894 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:28.796830 (+ 5936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:28.797172 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:28.797173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:28.797176 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:28.797239 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.797241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.798496 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.798497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.799555 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:28.799559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.800728 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.800729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:28.800813 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.800817 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:57:28.800867 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:57:28.800914 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4680,"thread_start_us":116,"threads_started":1}
I20260808 22:57:28.801316 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:28.786601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.1.212.1:36531)
0808 22:57:28.786764 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:28.786771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:28.790984 (+ 4213us) server_negotiation.cc:407] Connection header received
0808 22:57:28.796967 (+ 5983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:28.796971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:28.796990 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:28.797047 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:28.797324 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.798372 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.799724 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:28.800548 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:28.800601 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:28.800657 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:28.801179 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 22:57:28.801235 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260808 22:57:28.801733 16257 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: No bootstrap required, opened a new log
I20260808 22:57:28.801795 16257 ts_tablet_manager.cc:1397] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:57:28.802024 15871 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a (DEFAULT_TABLE table=test-table [id=4ce53f08e56845a89e42c676e16d0b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:28.802366 15871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafb13035d24ce0a03d1740ff225a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:28.804563 16257 raft_consensus.cc:348] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.805195 16257 raft_consensus.cc:374] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:28.805243 16257 raft_consensus.cc:729] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd59636f65d4d9681388d499d6abc5f, State: Initialized, Role: FOLLOWER
I20260808 22:57:28.805392 16257 consensus_queue.cc:260] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.805687 16257 ts_tablet_manager.cc:1428] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 22:57:28.805811 16224 heartbeater.cc:503] Master 127.1.212.62:43565 was elected leader, sending a full tablet report...
I20260808 22:57:28.806102 16261 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Bootstrap starting.
I20260808 22:57:28.806823 16259 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Bootstrap starting.
I20260808 22:57:28.806864 16261 tablet_bootstrap.cc:654] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:28.807381 16261 log.cc:824] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:28.808126 16259 tablet_bootstrap.cc:654] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:28.808490 16259 log.cc:824] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:28.809293 16259 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: No bootstrap required, opened a new log
I20260808 22:57:28.809389 16259 ts_tablet_manager.cc:1397] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:28.809477 16261 tablet_bootstrap.cc:492] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: No bootstrap required, opened a new log
I20260808 22:57:28.809542 16261 ts_tablet_manager.cc:1397] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:57:28.812013 16259 raft_consensus.cc:348] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.812187 16259 raft_consensus.cc:374] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:28.812215 16259 raft_consensus.cc:729] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2cd62cf058147c2ae6c14be2e6d8c89, State: Initialized, Role: FOLLOWER
I20260808 22:57:28.812233 16261 raft_consensus.cc:348] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.812395 16261 raft_consensus.cc:374] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:28.812431 16261 raft_consensus.cc:729] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c3a9ad1fb5419297dcc0b18f0a4b7f, State: Initialized, Role: FOLLOWER
I20260808 22:57:28.812433 16259 consensus_queue.cc:260] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.812532 16261 consensus_queue.cc:260] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:28.812723 16259 ts_tablet_manager.cc:1428] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:28.812860 16080 heartbeater.cc:503] Master 127.1.212.62:43565 was elected leader, sending a full tablet report...
I20260808 22:57:28.812844 16261 ts_tablet_manager.cc:1428] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:28.813218 15940 heartbeater.cc:503] Master 127.1.212.62:43565 was elected leader, sending a full tablet report...
Aug 08 22:57:28 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
W20260808 22:57:28.846174 15941 tablet.cc:2369] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:57:28.992507 16225 tablet.cc:2369] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:57:29.036022 16081 tablet.cc:2369] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:57:29 dist-test-slave-1x4g krb5kdc[14672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for HTTP/127.1.212.20@KRBTEST.COM
I20260808 22:57:29.107204 16266 raft_consensus.cc:482] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:29.107369 16266 raft_consensus.cc:504] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:29.107829 16266 leader_election.cc:288] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fd59636f65d4d9681388d499d6abc5f (127.1.212.3:37879), e2cd62cf058147c2ae6c14be2e6d8c89 (127.1.212.2:40975)
I20260808 22:57:29.114238 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:29.108310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:35595 (local address 127.1.212.3:37879)
0808 22:57:29.108366 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:29.108369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:29.110466 (+ 2097us) server_negotiation.cc:407] Connection header received
0808 22:57:29.110588 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:29.110590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:29.110604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:29.110635 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:29.110946 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.111675 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.113174 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.113756 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.113808 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:29.113821 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:29.114104 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 22:57:29.114153 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:57:29.114694 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:29.108044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:37879 (local address 127.1.212.1:35595)
0808 22:57:29.110360 (+ 2316us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:29.110386 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:29.110504 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:29.110743 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:29.110744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:29.110746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:29.110841 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:29.110844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.111829 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.111830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:29.113058 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:29.113062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.113938 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.113939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:29.114011 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:29.114015 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:57:29.114496 (+ 481us) client_negotiation.cc:240] Negotiation successful
0808 22:57:29.114550 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2289,"thread_start_us":112,"threads_started":1}
I20260808 22:57:29.115208 16175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafb13035d24ce0a03d1740ff225a8a" candidate_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd59636f65d4d9681388d499d6abc5f" is_pre_election: true
I20260808 22:57:29.115407 16175 raft_consensus.cc:2454] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c3a9ad1fb5419297dcc0b18f0a4b7f in term 0.
I20260808 22:57:29.115723 15825 leader_election.cc:302] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd59636f65d4d9681388d499d6abc5f, 84c3a9ad1fb5419297dcc0b18f0a4b7f; no voters:
I20260808 22:57:29.115928 16266 raft_consensus.cc:2781] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:57:29.116034 16266 raft_consensus.cc:482] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:57:29.116072 16266 raft_consensus.cc:3037] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:29.117143 16266 raft_consensus.cc:504] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:29.117314 16266 leader_election.cc:288] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [CANDIDATE]: Term 1 election: Requested vote from peers 7fd59636f65d4d9681388d499d6abc5f (127.1.212.3:37879), e2cd62cf058147c2ae6c14be2e6d8c89 (127.1.212.2:40975)
I20260808 22:57:29.117583 16175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafb13035d24ce0a03d1740ff225a8a" candidate_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd59636f65d4d9681388d499d6abc5f"
I20260808 22:57:29.117693 16175 raft_consensus.cc:3037] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:29.118815 16175 raft_consensus.cc:2454] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c3a9ad1fb5419297dcc0b18f0a4b7f in term 1.
I20260808 22:57:29.119036 15825 leader_election.cc:302] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd59636f65d4d9681388d499d6abc5f, 84c3a9ad1fb5419297dcc0b18f0a4b7f; no voters:
I20260808 22:57:29.119860 16266 raft_consensus.cc:2781] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:29.120071 16266 raft_consensus.cc:686] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [term 1 LEADER]: Becoming Leader. State: Replica: 84c3a9ad1fb5419297dcc0b18f0a4b7f, State: Running, Role: LEADER
I20260808 22:57:29.120193 16266 consensus_queue.cc:237] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } }
I20260808 22:57:29.121172 15731 catalog_manager.cc:5495] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c3a9ad1fb5419297dcc0b18f0a4b7f (127.1.212.1). New cstate: current_term: 1 leader_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 } health_report { overall_health: UNKNOWN } } }
I20260808 22:57:29.128829 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:29.108533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:41435 (local address 127.1.212.2:40975)
0808 22:57:29.108579 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:29.108581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:29.111141 (+ 2560us) server_negotiation.cc:407] Connection header received
0808 22:57:29.111287 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:29.111288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:29.111301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:29.111330 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:29.125598 (+ 14268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.126361 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.127783 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.128351 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.128436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:29.128451 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:29.128673 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 22:57:29.128729 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:57:29.128819 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:29.108335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:40975 (local address 127.1.212.1:41435)
0808 22:57:29.110741 (+ 2406us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:29.110753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:29.111193 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:29.123687 (+ 12494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:29.123689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:29.123692 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:29.125465 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:29.125471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.126525 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.126526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:29.127692 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:29.127696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.128491 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.128492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:29.128575 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:29.128581 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:57:29.128645 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:57:29.128696 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2384,"thread_start_us":74,"threads_started":1}
I20260808 22:57:29.129519 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafb13035d24ce0a03d1740ff225a8a" candidate_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" is_pre_election: true
I20260808 22:57:29.129588 16030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafb13035d24ce0a03d1740ff225a8a" candidate_uuid: "84c3a9ad1fb5419297dcc0b18f0a4b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89"
I20260808 22:57:29.129696 16031 raft_consensus.cc:2454] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c3a9ad1fb5419297dcc0b18f0a4b7f in term 0.
I20260808 22:57:29.155942 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:29.152324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.1.212.1:36531)
0808 22:57:29.152381 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:29.152384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:29.152396 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:57:29.152494 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:29.152495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:29.152521 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:29.152587 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:57:29.152810 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.153543 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.154669 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:29.154957 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:29.155045 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:29.155236 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:57:29.155381 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:57:29.155478 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:29.155801 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 22:57:29.155828 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:57:29.158295 16030 raft_consensus.cc:3037] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:29.159384 16030 raft_consensus.cc:1261] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 1 FOLLOWER]: Refusing update from remote peer 84c3a9ad1fb5419297dcc0b18f0a4b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:29.159869 16266 consensus_queue.cc:1035] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2cd62cf058147c2ae6c14be2e6d8c89" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:29.160718 16175 raft_consensus.cc:1261] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 1 FOLLOWER]: Refusing update from remote peer 84c3a9ad1fb5419297dcc0b18f0a4b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:29.161454 16266 consensus_queue.cc:1035] T 6bafb13035d24ce0a03d1740ff225a8a P 84c3a9ad1fb5419297dcc0b18f0a4b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd59636f65d4d9681388d499d6abc5f" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:29.169123 16274 mvcc.cc:205] Tried to move back new op lower bound from 7316397462144790528 to 7316397461992321024. Current Snapshot: MvccSnapshot[applied={T|T < 7316397462144790528}]
I20260808 22:57:29.171016 16275 mvcc.cc:205] Tried to move back new op lower bound from 7316397462144790528 to 7316397461992321024. Current Snapshot: MvccSnapshot[applied={T|T < 7316397462144790528}]
I20260808 22:57:29.171275 16277 mvcc.cc:205] Tried to move back new op lower bound from 7316397462144790528 to 7316397461992321024. Current Snapshot: MvccSnapshot[applied={T|T < 7316397462144790528}]
I20260808 22:57:29.174633 15731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:29.174750 15731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:29.178455 15731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7e3a1a22cd64d81a2c45f6d4d8d6ea1: Sending DeleteTablet for 3 replicas of tablet 6bafb13035d24ce0a03d1740ff225a8a
I20260808 22:57:29.178969 16008 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:29 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:50036
I20260808 22:57:29.179502 16138 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:29 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:50550
I20260808 22:57:29.180220 15871 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafb13035d24ce0a03d1740ff225a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:29 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:41500
I20260808 22:57:29.180711 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15773
I20260808 22:57:29.196935 16283 tablet_replica.cc:330] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: stopping tablet replica
I20260808 22:57:29.197131 16283 raft_consensus.cc:2229] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:29.197281 16283 raft_consensus.cc:2258] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:29.202977 16284 tablet_replica.cc:330] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: stopping tablet replica
I20260808 22:57:29.203166 16284 raft_consensus.cc:2229] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:29.203311 16284 raft_consensus.cc:2258] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:29.204439 16283 ts_tablet_manager.cc:1905] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:29.207561 16283 ts_tablet_manager.cc:1918] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:29.207664 16283 log.cc:1196] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/6bafb13035d24ce0a03d1740ff225a8a
I20260808 22:57:29.207960 16283 ts_tablet_manager.cc:1939] T 6bafb13035d24ce0a03d1740ff225a8a P e2cd62cf058147c2ae6c14be2e6d8c89: Deleting consensus metadata
I20260808 22:57:29.208474 15717 catalog_manager.cc:4841] TS e2cd62cf058147c2ae6c14be2e6d8c89 (127.1.212.2:40975): tablet 6bafb13035d24ce0a03d1740ff225a8a (table test-table [id=4ce53f08e56845a89e42c676e16d0b3f]) successfully deleted
W20260808 22:57:29.208735 15714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:36531 (error 108)
I20260808 22:57:29.209406 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15944
I20260808 22:57:29.214888 16284 ts_tablet_manager.cc:1905] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:29.217085 16284 ts_tablet_manager.cc:1918] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:29.217209 16284 log.cc:1196] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/6bafb13035d24ce0a03d1740ff225a8a
I20260808 22:57:29.217622 16284 ts_tablet_manager.cc:1939] T 6bafb13035d24ce0a03d1740ff225a8a P 7fd59636f65d4d9681388d499d6abc5f: Deleting consensus metadata
I20260808 22:57:29.218323 15716 catalog_manager.cc:4841] TS 7fd59636f65d4d9681388d499d6abc5f (127.1.212.3:37879): tablet 6bafb13035d24ce0a03d1740ff225a8a (table test-table [id=4ce53f08e56845a89e42c676e16d0b3f]) successfully deleted
I20260808 22:57:29.224871 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16084
I20260808 22:57:29.247855 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 15696
I20260808 22:57:29.255999 1872 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:57:29.601297 1872 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:57:29.601387 1872 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:57:29.957893 1872 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:57:29.958 UTC [14706] LOG: received smart shutdown request
2026-08-08 22:57:29.963 UTC [14706] LOG: background worker "logical replication launcher" (PID 14715) exited with exit code 1
2026-08-08 22:57:29.963 UTC [14710] LOG: shutting down
2026-08-08 22:57:30.160 UTC [14706] LOG: database system is shut down
2026-08-08T22:57:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53094 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:57:30 dist-test-slave-1x4g krb5kdc[16295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:57:30 dist-test-slave-1x4g krb5kdc[16295](info): set up 2 sockets
Aug 08 22:57:30 dist-test-slave-1x4g krb5kdc[16295](info): commencing operation
krb5kdc: starting...
W20260808 22:57:31.377503 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:57:31.515388 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:32945
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:31.604149 16309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:31.604475 16309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:31.604522 16309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:31.605863 16309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:31.605931 16309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:31.605967 16309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:31.605999 16309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:31.606029 16309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:31.607661 16309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:32945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:31.607969 16309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:31.608228 16309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:31.613665 16315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:31.613669 16314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:31.613663 16318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:31.613905 16309 server_base.cc:1034] running on GCE node
I20260808 22:57:31.614167 16309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:31.614477 16309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:31.615662 16309 hybrid_clock.cc:648] HybridClock initialized: now 1786229851615650 us; error 34 us; skew 500 ppm
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:31.618664 16309 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:31.619196 16309 webserver.cc:460] Webserver started at http://127.1.212.62:41105/ using document root <none> and password file <none>
I20260808 22:57:31.619488 16309 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:31.619565 16309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:31.619724 16309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:31.621006 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "170fefd89c134e4989a8744a730db2e5"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "7a5ee3a0fe07622c1e675e28e4dc1dce"
server_key_iv: "610fc773ac20b53e0423ad0956f2085c"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.621415 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "170fefd89c134e4989a8744a730db2e5"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "7a5ee3a0fe07622c1e675e28e4dc1dce"
server_key_iv: "610fc773ac20b53e0423ad0956f2085c"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.622936 16309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:57:31.623771 16325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:31.624001 16309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:31.624125 16309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "170fefd89c134e4989a8744a730db2e5"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "7a5ee3a0fe07622c1e675e28e4dc1dce"
server_key_iv: "610fc773ac20b53e0423ad0956f2085c"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.624217 16309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:31.634709 16309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:31.637298 16309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:31.637492 16309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:31.641966 16309 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:32945
I20260808 22:57:31.642009 16380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:32945 every 8 connection(s)
I20260808 22:57:31.642419 16309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:31.643544 16381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:31.645370 16381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:31.652102 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16309
I20260808 22:57:31.652232 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:57:31.652441 1872 external_mini_cluster.cc:1428] Setting key 5074c98ad42d4806344d7402cef637e4
I20260808 22:57:31.655965 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.646156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:32945 (local address 127.0.0.1:35000)
0808 22:57:31.646341 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.646388 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:31.646882 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:31.647171 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:31.647173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:31.647448 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:31.647632 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.647636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.648706 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.648707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.649512 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.649515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.650039 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.650040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.650103 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.650628 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:31.650638 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:31.652114 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:31.654210 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.654220 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.654223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.654472 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.654788 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.654789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.654790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.654911 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.655428 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:31.655435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:31.655515 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 22:57:31.655598 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:57:31.655757 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260808 22:57:31.656009 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.646223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.1.212.62:32945)
0808 22:57:31.646393 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:31.646398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:31.646489 (+ 91us) server_negotiation.cc:407] Connection header received
0808 22:57:31.647027 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:31.647029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:31.647047 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:31.647138 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:31.647740 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.648605 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.649611 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.649948 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.650047 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.652253 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:31.652259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:31.652260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:31.652267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:31.654087 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.654565 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.654569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.654569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.654693 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.654988 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.654989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.654989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.655340 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:31.655439 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:31.655661 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 22:57:31.655852 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260808 22:57:31.656872 16381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:31.657276 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.657150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43206)
0808 22:57:31.657204 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.657252 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:57:31.657450 16327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
I20260808 22:57:31.657917 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.657808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43220)
0808 22:57:31.657852 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.657897 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 22:57:31.658063 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:44963: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.212.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
I20260808 22:57:31.683617 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:44963
--webserver_interface=127.1.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:31.707353 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } attempt: 1
I20260808 22:57:31.707960 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.707814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43226)
0808 22:57:31.707882 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.707924 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:57:31.708427 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.708278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43230)
0808 22:57:31.708345 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.708394 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:57:31.708581 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:44963: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
W20260808 22:57:31.773772 16388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:31.774021 16388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:31.774047 16388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:31.775353 16388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:31.775415 16388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:31.775430 16388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:31.775450 16388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:31.775463 16388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:31.776808 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } attempt: 2
I20260808 22:57:31.777348 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.777167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43246)
0808 22:57:31.777248 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.777305 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:57:31.777458 16388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:44963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:31.777865 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.777750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43252)
0808 22:57:31.777794 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.777835 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 22:57:31.777909 16388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260808 22:57:31.778110 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:44963: Network error: Client connection negotiation failed: client connection to 127.1.212.61:44963: connect: Connection refused (error 111)
I20260808 22:57:31.778232 16388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:31.783427 16393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:31.783418 16394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:31.783512 16388 server_base.cc:1034] running on GCE node
W20260808 22:57:31.783424 16396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:31.783990 16388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:31.784353 16388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:31.785514 16388 hybrid_clock.cc:648] HybridClock initialized: now 1786229851785503 us; error 33 us; skew 500 ppm
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:31.788275 16388 init.cc:377] Logged in from keytab as kudu/127.1.212.61@KRBTEST.COM (short username kudu)
I20260808 22:57:31.788895 16388 webserver.cc:460] Webserver started at http://127.1.212.61:45113/ using document root <none> and password file <none>
I20260808 22:57:31.789228 16388 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:31.789305 16388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:31.789459 16388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:31.790640 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data/instance:
uuid: "7d9ded35d6094943a9e268e2270efc65"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "f97da9b5328e0993e977b1191a80fe44"
server_key_iv: "ab9cfd73d655d126a3cb1388a950e8e2"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.791033 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal/instance:
uuid: "7d9ded35d6094943a9e268e2270efc65"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "f97da9b5328e0993e977b1191a80fe44"
server_key_iv: "ab9cfd73d655d126a3cb1388a950e8e2"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.792552 16388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:57:31.793538 16403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:31.793735 16388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:31.793855 16388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "7d9ded35d6094943a9e268e2270efc65"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "f97da9b5328e0993e977b1191a80fe44"
server_key_iv: "ab9cfd73d655d126a3cb1388a950e8e2"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.793944 16388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:31.819502 16388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:31.821983 16388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:31.822170 16388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:31.826988 16388 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.61:44963
I20260808 22:57:31.827040 16457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.61:44963 every 8 connection(s)
I20260808 22:57:31.827515 16388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
I20260808 22:57:31.828621 16458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:31.830381 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16388
I20260808 22:57:31.830518 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-1/wal/instance
I20260808 22:57:31.830721 1872 external_mini_cluster.cc:1428] Setting key d357839f18a423b9c35d9b3330aad46e
I20260808 22:57:31.830735 16458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.212.60@KRBTEST.COM; defaulting to no policy
I20260808 22:57:31.841585 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.832648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.1.212.62:32945)
0808 22:57:31.832723 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:31.832725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:31.832734 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:31.832760 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:31.832761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:31.832818 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:31.832897 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:31.833625 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.834137 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.835175 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.835404 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.835482 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.838202 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:31.838204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:31.838204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:31.838220 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:31.839982 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.840474 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.840476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.840477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.840547 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.840863 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.840864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.840864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.841005 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:31.841080 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:31.841383 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 22:57:31.841486 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:57:31.841812 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.831652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:32945 (local address 127.0.0.1:35012)
0808 22:57:31.831845 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.831911 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:31.832385 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:31.833007 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:31.833012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:31.833297 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:31.833518 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.833522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.834249 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.834251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.835075 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.835078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.835493 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.835494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.835636 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.836252 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:31.836262 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:31.838061 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:31.840100 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.840114 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.840117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.840386 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.840616 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.840618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.840618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.840746 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.841071 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:31.841080 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:31.841240 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 22:57:31.841351 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 22:57:31.841550 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.60@KRBTEST.COM" created.
I20260808 22:57:31.842479 16458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:57:31.850749 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.842857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.1.212.61:44963)
0808 22:57:31.843033 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:31.843037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:31.843056 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:57:31.843074 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:31.843075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:31.843087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:31.843164 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:31.843455 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.844268 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.845097 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.845612 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.845691 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.847729 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:31.847742 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:31.847742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:31.847749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:31.849459 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.849902 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.849906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.849906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.849963 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.850168 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.850169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.850169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.850319 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:31.850380 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:31.850567 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 22:57:31.850661 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260808 22:57:31.850894 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.842770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43264)
0808 22:57:31.842822 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.842832 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:31.842870 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:31.843165 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:31.843166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:31.843316 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:31.843373 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.843375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.844404 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.844405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.845012 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.845015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.845698 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.845699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.845769 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.846248 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:31.846259 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:31.847630 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:31.849557 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.849559 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.849559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.849810 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.850030 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.850031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.850031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.850099 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.850390 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:31.850391 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:31.850452 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 22:57:31.850538 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 22:57:31.850726 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":62}
I20260808 22:57:31.851613 16458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:31.852043 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.851906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55834)
0808 22:57:31.851957 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.852020 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 22:57:31.852185 16407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
I20260808 22:57:31.852634 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.852538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55838)
0808 22:57:31.852586 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.852610 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 22:57:31.852763 16458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
I20260808 22:57:31.860360 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:45183
--webserver_interface=127.1.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:31.869284 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } attempt: 3
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:57:31.876986 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.869765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.1.212.61:44963)
0808 22:57:31.869839 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:31.869841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:31.869854 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:57:31.869912 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:31.869913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:31.869931 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:31.869991 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:31.870349 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.870907 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.871614 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.871826 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.871921 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.874236 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:31.874238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:31.874238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:31.874253 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:31.875683 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.876080 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.876081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.876081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.876147 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:31.876353 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:31.876355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:31.876355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:31.876491 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:31.876571 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:31.876746 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 22:57:31.876890 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:57:31.876986 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.869687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43278)
0808 22:57:31.869758 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.869770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:31.869840 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:31.869998 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:31.869999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:31.870186 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:31.870260 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.870262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.871013 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.871014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.871533 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:31.871534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:31.871931 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:31.871932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:31.872024 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:31.872576 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:31.872582 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:31.874132 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:31.875794 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.875795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.875796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.875995 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.876218 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:31.876219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:31.876220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:31.876289 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:31.876562 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:31.876563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:31.876626 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 22:57:31.876702 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:57:31.876874 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:57:31.877543 16381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:31.877936 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.877823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55848)
0808 22:57:31.877873 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.877904 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 22:57:31.878273 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.878183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55854)
0808 22:57:31.878225 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.878250 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 22:57:31.878391 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
I20260808 22:57:31.902127 16458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 1
I20260808 22:57:31.902734 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.902582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55858)
0808 22:57:31.902651 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.902700 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:57:31.903180 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.903091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55866)
0808 22:57:31.903136 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.903162 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:57:31.903297 16458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
I20260808 22:57:31.909559 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 1
I20260808 22:57:31.910058 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.909921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55876)
0808 22:57:31.909987 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.910026 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:57:31.910401 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.910305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55882)
0808 22:57:31.910347 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.910381 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 22:57:31.910493 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
W20260808 22:57:31.951299 16467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:31.951597 16467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:31.951644 16467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:31.953033 16467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:31.953117 16467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:31.953145 16467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:31.953169 16467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:31.953198 16467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:31.954887 16467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:32945,127.1.212.61:44963,127.1.212.60:45183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:45183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:31.955216 16467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:31.955472 16467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:31.960649 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:31.960657 16475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:31.960737 16467 server_base.cc:1034] running on GCE node
W20260808 22:57:31.960657 16472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:31.961226 16467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:31.961557 16467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:31.962725 16467 hybrid_clock.cc:648] HybridClock initialized: now 1786229851962708 us; error 39 us; skew 500 ppm
Aug 08 22:57:31 dist-test-slave-1x4g krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:31.965447 16467 init.cc:377] Logged in from keytab as kudu/127.1.212.60@KRBTEST.COM (short username kudu)
I20260808 22:57:31.965963 16467 webserver.cc:460] Webserver started at http://127.1.212.60:37199/ using document root <none> and password file <none>
I20260808 22:57:31.966231 16467 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:31.966305 16467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:31.966574 16467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:31.967742 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data/instance:
uuid: "db3bd6d3fa14445189bf26102be90c3d"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "11ece4c201e3d0522dc6127cfc08e157"
server_key_iv: "735fb545e33acd94daf7dbea487b7956"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.968149 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal/instance:
uuid: "db3bd6d3fa14445189bf26102be90c3d"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "11ece4c201e3d0522dc6127cfc08e157"
server_key_iv: "735fb545e33acd94daf7dbea487b7956"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.969734 16467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:57:31.970609 16483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:31.970793 16467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:31.970907 16467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "db3bd6d3fa14445189bf26102be90c3d"
format_stamp: "Formatted at 2026-08-08 22:57:31 on dist-test-slave-1x4g"
server_key: "11ece4c201e3d0522dc6127cfc08e157"
server_key_iv: "735fb545e33acd94daf7dbea487b7956"
server_key_version: "encryptionkey@0"
I20260808 22:57:31.970993 16467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:31.971475 16458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 2
I20260808 22:57:31.971946 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.971784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55898)
0808 22:57:31.971854 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.971905 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:57:31.972354 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.972249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55908)
0808 22:57:31.972298 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.972329 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:57:31.972489 16458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
I20260808 22:57:31.985641 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 2
I20260808 22:57:31.986153 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.986016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55924)
0808 22:57:31.986083 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.986122 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:57:31.986604 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:31.986502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55928)
0808 22:57:31.986553 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:31.986582 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:57:31.986697 16381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:45183: Network error: Client connection negotiation failed: client connection to 127.1.212.60:45183: connect: Connection refused (error 111)
I20260808 22:57:31.993121 16467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:31.995208 16467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:31.995380 16467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:31.999819 16467 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.60:45183
I20260808 22:57:31.999871 16536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.60:45183 every 8 connection(s)
I20260808 22:57:32.000281 16467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
I20260808 22:57:32.001243 16537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:32.002913 16537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:32.006953 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16467
I20260808 22:57:32.007087 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229688278322-1872-0/minicluster-data/master-2/wal/instance
I20260808 22:57:32.007265 1872 external_mini_cluster.cc:1428] Setting key 3bc6cee82bc9fa7807ec3856d622cb7d
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:57:32.013633 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.003761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.1.212.62:32945)
0808 22:57:32.003843 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.003846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.004043 (+ 197us) server_negotiation.cc:407] Connection header received
0808 22:57:32.004511 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.004512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.004522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.004576 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.005291 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.005771 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.006880 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.007219 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.007304 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.009698 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.009700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.009700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.009710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.012022 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.012491 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.012492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.012493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.012564 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.012884 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.012885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.012885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.013041 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.013109 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.013487 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.013563 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:57:32.013870 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.003711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:32945 (local address 127.0.0.1:35016)
0808 22:57:32.003908 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:32.003961 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:32.004422 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:32.004641 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:32.004645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:32.004978 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:32.005189 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.005194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.005858 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.005858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.006688 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.006691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.007323 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.007325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.007499 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.008100 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:32.008124 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:32.009592 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:32.012120 (+ 2528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.012128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.012130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.012408 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.012643 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.012643 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.012644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.012752 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.013120 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:32.013130 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:32.013305 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 22:57:32.013446 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:57:32.013612 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260808 22:57:32.014633 16537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:32.017510 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.007712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.1.212.60:45183)
0808 22:57:32.007896 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.007900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.008250 (+ 350us) server_negotiation.cc:407] Connection header received
0808 22:57:32.008316 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.008319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.008345 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.008432 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.009089 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.009831 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.010966 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.011402 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.011462 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.014192 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.014201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.014201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.014208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.016228 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.016656 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.016659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.016659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.016714 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.016939 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.016940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.016940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.017075 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.017216 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.017303 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.017398 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:32.023233 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.015021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.1.212.61:44963)
0808 22:57:32.015064 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.015066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.015076 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:57:32.015106 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.015107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.015119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.015179 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.015472 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.016228 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.017219 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.017504 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.017581 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.019762 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.019763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.019764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.019772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.022123 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.022463 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.022465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.022465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.022529 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.022752 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.022753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.022753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.022859 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.022911 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.023075 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.023134 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:57:32.023273 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.014959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:44963 (local address 127.0.0.1:43290)
0808 22:57:32.015008 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:32.015019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:32.015058 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:32.015181 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:32.015182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:32.015351 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:32.015394 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.015396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.016325 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.016326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.017136 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.017139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.017586 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.017587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.017668 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.018137 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:32.018142 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:32.019652 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:32.022211 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.022212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.022212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.022390 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.022607 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.022608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.022608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.022681 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.022924 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:32.022925 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:32.022978 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 22:57:32.023059 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:57:32.023156 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 22:57:32.023749 16537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:57:32.026515 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.018345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.1.212.62:32945)
0808 22:57:32.018396 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.018398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.018407 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:32.018432 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.018440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.018450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.018507 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.019149 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.019807 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.020849 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.021102 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.021159 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.023722 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.023723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.023724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.023732 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.025274 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.025643 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.025645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.025645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.025711 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.025892 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.025893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.025893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.026026 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.026235 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.026276 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.026396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:57:32.031457 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.024083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.1.212.60:45183)
0808 22:57:32.024136 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.024138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.024147 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:32.024233 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.024234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.024245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.024276 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.024823 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.025348 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.026215 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.026506 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.026561 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.028699 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.028701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.028701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.028709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.030256 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.030686 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.030687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.030687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.030740 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.030971 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.030972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.030972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.031088 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.031147 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.031319 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.031392 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:57:32.031597 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.024050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55954)
0808 22:57:32.024082 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:32.024091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:32.024175 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:32.024541 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:32.024542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:32.024646 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:32.024710 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.024712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.025449 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.025450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.026138 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.026141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.026576 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.026577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.026641 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.027125 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:32.027132 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:32.028619 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:32.030337 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.030339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.030340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.030608 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.030807 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.030808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.030808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.030879 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.031158 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:32.031159 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:32.031214 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 22:57:32.031305 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:57:32.031457 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":48}
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:57:32.033419 16537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: Bootstrap starting.
I20260808 22:57:32.034473 16537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:32.034776 16537 log.cc:824] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:32.035609 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.027190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.1.212.61:44963)
0808 22:57:32.027230 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.027232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.027244 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:57:32.027270 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.027271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.027284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.027324 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.028022 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.028898 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.029718 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.030036 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.030119 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.032928 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.032930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.032930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.032938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.034434 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.034794 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.034795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.034796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.034856 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.035097 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.035098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.035099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.035255 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.035446 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.035482 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.035546 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:57:32.035702 16537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: No bootstrap required, opened a new log
I20260808 22:57:32.037108 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:57:32.037496 16537 raft_consensus.cc:348] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.037650 16537 raft_consensus.cc:374] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:32.037714 16537 raft_consensus.cc:729] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3bd6d3fa14445189bf26102be90c3d, State: Initialized, Role: FOLLOWER
I20260808 22:57:32.037858 16537 consensus_queue.cc:260] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.038223 16537 sys_catalog.cc:564] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:32.038179 16543 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.038290 16543 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:57:32.040248 16561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:57:32.040374 16561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:57:32.041395 16467 master_runner.cc:186] Error getting master registration for 127.1.212.62:32945: OK, instance_id { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" instance_seqno: 1786229851640602 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:57:32.044888 16467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:57:32.047127 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.038620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.1.212.61:44963)
0808 22:57:32.038663 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.038665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.038801 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:57:32.038903 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.038904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.038917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.038966 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.039976 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.040732 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.041716 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.042005 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.042071 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.043889 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.043891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.043891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.043899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.045548 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.045933 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.045934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.045934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.045997 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.046206 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.046207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.046208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.046337 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.046454 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.046909 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.047003 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:57:32.048036 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.038741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.1.212.62:32945)
0808 22:57:32.038794 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.038797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.039028 (+ 231us) server_negotiation.cc:407] Connection header received
0808 22:57:32.039140 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.039141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.039153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.039184 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.040066 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.040936 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.041771 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.042076 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.042116 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.043608 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.043610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.043610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.043618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.045846 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.047111 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.047112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.047113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.047177 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.047498 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.047499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.047499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.047622 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.047721 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.047831 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.047916 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:57:32.050937 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.039101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.1.212.60:45183)
0808 22:57:32.039141 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.039142 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.042530 (+ 3388us) server_negotiation.cc:407] Connection header received
0808 22:57:32.042586 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.042587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.042600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.042630 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.044179 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.044960 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.046004 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.046305 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.046382 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.048086 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.048087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.048088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.048096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.049678 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.050037 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.050039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.050039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.050102 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.050355 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.050356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.050356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.050549 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.050647 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.050759 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.050843 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:57:32.063663 16458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 3
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:57:32.071448 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.064099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55980)
0808 22:57:32.064170 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:32.064184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:32.064264 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:32.064421 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:32.064422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:32.064606 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:32.064654 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.064655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.065409 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.065411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.065904 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.065906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.066361 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.066362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.066512 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.067217 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:32.067224 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:32.068520 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:32.070302 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.070303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.070304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.070513 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.070735 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.070736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.070737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.070804 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.071077 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:32.071078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:32.071129 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:57:32.071209 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:57:32.071332 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 22:57:32.071439 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.064210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.1.212.60:45183)
0808 22:57:32.064274 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.064276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.064289 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:57:32.064333 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.064334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.064349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.064411 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.064748 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.065322 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.065995 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.066226 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.066292 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.068631 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.068633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.068633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.068641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.070191 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.070606 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.070607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.070608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.070667 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.070866 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.070867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.070868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.071001 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.071072 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.071237 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.071335 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:57:32.073115 16458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: Bootstrap starting.
I20260808 22:57:32.073794 16458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:32.074165 16458 log.cc:824] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:32.075116 16458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: No bootstrap required, opened a new log
I20260808 22:57:32.076767 16458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.076974 16458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:32.076998 16458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d9ded35d6094943a9e268e2270efc65, State: Initialized, Role: FOLLOWER
I20260808 22:57:32.077145 16458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.077441 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.077513 16458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:32.077529 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:57:32.079396 16575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:57:32.079469 16575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:57:32.079833 16388 master_runner.cc:186] Error getting master registration for 127.1.212.62:32945: OK, instance_id { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" instance_seqno: 1786229851640602 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:57:32.082893 16388 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:57:32.085860 16381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } attempt: 3
Aug 08 22:57:32 dist-test-slave-1x4g krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:57:32.095283 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.086197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:45183 (local address 127.0.0.1:55988)
0808 22:57:32.086260 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:32.086273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:32.086316 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:32.086503 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:32.086504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:32.086709 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:32.086772 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.086775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.087728 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.087729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.088554 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:32.088558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.089085 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.089086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:32.089153 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.089712 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:32.089718 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:32.091987 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:32.093995 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.093996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.093997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.094249 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.094516 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:32.094517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:32.094517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:32.094598 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:32.094897 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:32.094898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:32.094957 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 22:57:32.095038 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:57:32.095167 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:57:32.095276 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:32.086279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.1.212.60:45183)
0808 22:57:32.086337 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:32.086339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:32.086352 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:57:32.086389 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:32.086390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:32.086401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:32.086475 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:32.086861 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.087625 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.088656 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:32.088979 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:32.089053 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:32.092089 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:32.092091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:32.092092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:32.092101 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:32.093887 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.094359 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.094360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.094360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.094440 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:32.094668 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:32.094669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:32.094669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:32.094823 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:32.094897 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:32.095074 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 22:57:32.095171 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:57:32.097604 16381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: Bootstrap starting.
I20260808 22:57:32.098465 16381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:32.098800 16381 log.cc:824] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:32.099839 16381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: No bootstrap required, opened a new log
I20260808 22:57:32.102141 16381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.102355 16381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:32.102443 16381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170fefd89c134e4989a8744a730db2e5, State: Initialized, Role: FOLLOWER
I20260808 22:57:32.102620 16381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.103025 16381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:32.104167 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.104281 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:57:32.105314 16591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:57:32.105384 16591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:57:32.108146 16309 master_runner.cc:263] No leader master found from master 170fefd89c134e4989a8744a730db2e5
I20260808 22:57:32.108218 16309 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 22:57:32.138386 16331 tablet.cc:2369] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:57:32.143131 16579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:32.143271 16579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.143532 16579 leader_election.cc:288] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d9ded35d6094943a9e268e2270efc65 (127.1.212.61:44963), db3bd6d3fa14445189bf26102be90c3d (127.1.212.60:45183)
I20260808 22:57:32.144021 16431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "170fefd89c134e4989a8744a730db2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d9ded35d6094943a9e268e2270efc65" is_pre_election: true
I20260808 22:57:32.144186 16431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170fefd89c134e4989a8744a730db2e5 in term 0.
I20260808 22:57:32.144680 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "170fefd89c134e4989a8744a730db2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3bd6d3fa14445189bf26102be90c3d" is_pre_election: true
I20260808 22:57:32.144883 16510 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170fefd89c134e4989a8744a730db2e5 in term 0.
I20260808 22:57:32.145169 16327 leader_election.cc:302] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170fefd89c134e4989a8744a730db2e5, 7d9ded35d6094943a9e268e2270efc65; no voters:
I20260808 22:57:32.145354 16579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:57:32.145401 16579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:57:32.145423 16579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:32.146377 16579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.146507 16579 leader_election.cc:288] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [CANDIDATE]: Term 1 election: Requested vote from peers 7d9ded35d6094943a9e268e2270efc65 (127.1.212.61:44963), db3bd6d3fa14445189bf26102be90c3d (127.1.212.60:45183)
I20260808 22:57:32.146898 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "170fefd89c134e4989a8744a730db2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3bd6d3fa14445189bf26102be90c3d"
I20260808 22:57:32.146999 16510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:32.147930 16510 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170fefd89c134e4989a8744a730db2e5 in term 1.
I20260808 22:57:32.148187 16329 leader_election.cc:302] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170fefd89c134e4989a8744a730db2e5, db3bd6d3fa14445189bf26102be90c3d; no voters:
I20260808 22:57:32.148453 16431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "170fefd89c134e4989a8744a730db2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d9ded35d6094943a9e268e2270efc65"
I20260808 22:57:32.148547 16431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:32.149423 16431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170fefd89c134e4989a8744a730db2e5 in term 1.
I20260808 22:57:32.149758 16579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:32.149885 16579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [term 1 LEADER]: Becoming Leader. State: Replica: 170fefd89c134e4989a8744a730db2e5, State: Running, Role: LEADER
I20260808 22:57:32.149983 16579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } }
I20260808 22:57:32.150396 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170fefd89c134e4989a8744a730db2e5. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.150470 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:32.150826 16596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:32.151072 16596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:32.154423 16510 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [term 1 FOLLOWER]: Refusing update from remote peer 170fefd89c134e4989a8744a730db2e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:32.154727 16431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [term 1 FOLLOWER]: Refusing update from remote peer 170fefd89c134e4989a8744a730db2e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:32.156047 16579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:32.157832 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170fefd89c134e4989a8744a730db2e5. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.157938 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:57:32.158138 16579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:32.159947 16543 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170fefd89c134e4989a8744a730db2e5. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.160075 16543 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:57:32.162744 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.162861 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:32.163511 16543 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.163610 16543 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:57:32.163909 16597 mvcc.cc:205] Tried to move back new op lower bound from 7316397474420436992 to 7316397474406985728. Current Snapshot: MvccSnapshot[applied={T|T < 7316397474420436992}]
I20260808 22:57:32.164535 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.164624 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:32.165261 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "170fefd89c134e4989a8744a730db2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170fefd89c134e4989a8744a730db2e5" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 32945 } } peers { permanent_uuid: "7d9ded35d6094943a9e268e2270efc65" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 44963 } } peers { permanent_uuid: "db3bd6d3fa14445189bf26102be90c3d" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 45183 } } }
I20260808 22:57:32.165359 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:57:32.165789 16601 mvcc.cc:205] Tried to move back new op lower bound from 7316397474420436992 to 7316397474406985728. Current Snapshot: MvccSnapshot[applied={T|T < 7316397474420436992}]
I20260808 22:57:32.165936 16598 mvcc.cc:205] Tried to move back new op lower bound from 7316397474420436992 to 7316397474406985728. Current Snapshot: MvccSnapshot[applied={T|T < 7316397474420436992}]
I20260808 22:57:32.167527 16596 catalog_manager.cc:1347] Generated new cluster ID: 8f46d4738a5143ec865d8272aab1bc52
I20260808 22:57:32.167588 16596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:32.188321 16596 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:57:32.190301 16596 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:32.206852 16596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 170fefd89c134e4989a8744a730db2e5: Generated new TSK 0
I20260808 22:57:32.207222 16596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:57:33.040979 16561 catalog_manager.cc:1259] Loaded cluster ID: 8f46d4738a5143ec865d8272aab1bc52
I20260808 22:57:33.041078 16561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: loading cluster ID for follower catalog manager: success
I20260808 22:57:33.041958 16561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: acquiring CA information for follower catalog manager: success
I20260808 22:57:33.042318 16561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db3bd6d3fa14445189bf26102be90c3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:57:33.079954 16575 catalog_manager.cc:1259] Loaded cluster ID: 8f46d4738a5143ec865d8272aab1bc52
I20260808 22:57:33.080039 16575 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: loading cluster ID for follower catalog manager: success
I20260808 22:57:33.080965 16575 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: acquiring CA information for follower catalog manager: success
I20260808 22:57:33.081313 16575 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d9ded35d6094943a9e268e2270efc65: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:57:33.274992 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16309
I20260808 22:57:33.282742 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16388
I20260808 22:57:33.291430 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16467
2026-08-08T22:57:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3008 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:57:33 dist-test-slave-1x4g krb5kdc[16613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:57:33 dist-test-slave-1x4g krb5kdc[16613](info): set up 2 sockets
Aug 08 22:57:33 dist-test-slave-1x4g krb5kdc[16613](info): commencing operation
krb5kdc: starting...
W20260808 22:57:34.366608 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:57:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:57:34.499694 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43425
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:34.589828 16628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:34.590112 16628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:34.590155 16628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:34.591528 16628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:34.591599 16628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:34.591625 16628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:34.591658 16628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:34.591691 16628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:34.593456 16628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:43425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:34.593773 16628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:34.594015 16628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:34.599316 16633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:34.599303 16636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:34.599282 16634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:34.599339 16628 server_base.cc:1034] running on GCE node
I20260808 22:57:34.599874 16628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:34.600162 16628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:34.601322 16628 hybrid_clock.cc:648] HybridClock initialized: now 1786229854601307 us; error 42 us; skew 500 ppm
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229854, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:34.604019 16628 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:34.604534 16628 webserver.cc:460] Webserver started at http://127.1.212.62:40975/ using document root <none> and password file <none>
I20260808 22:57:34.604854 16628 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:34.604934 16628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:34.605099 16628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:34.606277 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "2deb0537c1dc4766b57c6e7cb42e72d0"
format_stamp: "Formatted at 2026-08-08 22:57:34 on dist-test-slave-1x4g"
server_key: "2d122c162e231b77fca8648de4b73f1d"
server_key_iv: "d0492f329b05e416df4c741d00fbd82b"
server_key_version: "encryptionkey@0"
I20260808 22:57:34.606676 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "2deb0537c1dc4766b57c6e7cb42e72d0"
format_stamp: "Formatted at 2026-08-08 22:57:34 on dist-test-slave-1x4g"
server_key: "2d122c162e231b77fca8648de4b73f1d"
server_key_iv: "d0492f329b05e416df4c741d00fbd82b"
server_key_version: "encryptionkey@0"
I20260808 22:57:34.608093 16628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:57:34.608879 16643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:34.609052 16628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:34.609169 16628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "2deb0537c1dc4766b57c6e7cb42e72d0"
format_stamp: "Formatted at 2026-08-08 22:57:34 on dist-test-slave-1x4g"
server_key: "2d122c162e231b77fca8648de4b73f1d"
server_key_iv: "d0492f329b05e416df4c741d00fbd82b"
server_key_version: "encryptionkey@0"
I20260808 22:57:34.609246 16628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:34.623653 16628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:34.626205 16628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:34.626401 16628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:34.631000 16628 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:43425
I20260808 22:57:34.631098 16698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:43425 every 8 connection(s)
I20260808 22:57:34.631492 16628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:34.632503 16699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:34.635183 16699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0: Bootstrap starting.
I20260808 22:57:34.635840 16699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:34.636057 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16628
I20260808 22:57:34.636188 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:57:34.636276 16699 log.cc:824] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:34.636391 1872 external_mini_cluster.cc:1428] Setting key 0738063c0409315dd6824ea7ce9d1537
I20260808 22:57:34.637356 16699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0: No bootstrap required, opened a new log
I20260808 22:57:34.639662 16699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } }
I20260808 22:57:34.639854 16699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:34.639906 16699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2deb0537c1dc4766b57c6e7cb42e72d0, State: Initialized, Role: FOLLOWER
I20260808 22:57:34.640053 16699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } }
I20260808 22:57:34.640156 16699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:34.640203 16699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:34.640257 16699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:34.640987 16699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } }
I20260808 22:57:34.641115 16699 leader_election.cc:302] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2deb0537c1dc4766b57c6e7cb42e72d0; no voters:
I20260808 22:57:34.641315 16699 leader_election.cc:288] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:57:34.641381 16705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:34.641620 16699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:34.641628 16705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2deb0537c1dc4766b57c6e7cb42e72d0, State: Running, Role: LEADER
I20260808 22:57:34.641842 16705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } }
I20260808 22:57:34.642207 16706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } } }
I20260808 22:57:34.642343 16706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:34.642207 16707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2deb0537c1dc4766b57c6e7cb42e72d0. Latest consensus state: current_term: 1 leader_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2deb0537c1dc4766b57c6e7cb42e72d0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43425 } } }
I20260808 22:57:34.642584 16713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:34.642681 16707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:34.643247 16713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:34.645082 16713 catalog_manager.cc:1347] Generated new cluster ID: b2b8b10cb92d49b4bcc19ff670ba2f5d
I20260808 22:57:34.645141 16713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:34.648475 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:34.637042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.1.212.62:43425)
0808 22:57:34.637266 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:34.637277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:34.637317 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:57:34.637865 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:34.637873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:34.637931 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:34.638098 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:34.638787 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:34.639726 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:34.640677 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:34.641290 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:34.641428 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:34.644703 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:34.644708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:34.644709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:34.644722 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:34.646657 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:34.647063 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:34.647066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:34.647067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:34.647166 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:34.647415 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:34.647416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:34.647416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:34.647717 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:34.647821 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:34.648090 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 22:57:34.648248 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260808 22:57:34.649806 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:57:34.667114 16713 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:57:34.667697 16713 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:34.672930 16713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2deb0537c1dc4766b57c6e7cb42e72d0: Generated new TSK 0
I20260808 22:57:34.673178 16713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:57:34.703833 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16628
2026-08-08T22:57:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1411 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16728](info): set up 2 sockets
Aug 08 22:57:34 dist-test-slave-1x4g krb5kdc[16728](info): commencing operation
krb5kdc: starting...
W20260808 22:57:35.782701 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:57:35 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:57:35.919482 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:45295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:36.006690 16742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.006960 16742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.007002 16742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.008270 16742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:36.008352 16742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.008391 16742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.008421 16742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:36.008450 16742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:36.010164 16742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:45295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.010488 16742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.010738 16742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:36.016206 16752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.016208 16748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.016206 16749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.016889 16742 server_base.cc:1034] running on GCE node
I20260808 22:57:36.017175 16742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:36.017511 16742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:36.018689 16742 hybrid_clock.cc:648] HybridClock initialized: now 1786229856018670 us; error 45 us; skew 500 ppm
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:36.026326 16742 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:36.026921 16742 webserver.cc:460] Webserver started at http://127.1.212.62:41771/ using document root <none> and password file <none>
I20260808 22:57:36.027170 16742 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:36.027241 16742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:36.027416 16742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:36.028590 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "0dd0102b3e3246b79e6d4db3d64186e0"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "1e4f1e23c1c6379e9b5a0847b9dc4d66"
server_key_iv: "96e6dc4fce56723229a185536e250860"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.029022 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "0dd0102b3e3246b79e6d4db3d64186e0"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "1e4f1e23c1c6379e9b5a0847b9dc4d66"
server_key_iv: "96e6dc4fce56723229a185536e250860"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.030501 16742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:36.031280 16760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.031514 16742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:36.031627 16742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "0dd0102b3e3246b79e6d4db3d64186e0"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "1e4f1e23c1c6379e9b5a0847b9dc4d66"
server_key_iv: "96e6dc4fce56723229a185536e250860"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.031710 16742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:36.045002 16742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:36.047070 16742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:36.047251 16742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:36.051983 16742 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:45295
I20260808 22:57:36.052040 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:45295 every 8 connection(s)
I20260808 22:57:36.052454 16742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:36.053546 16816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:36.055761 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16742
I20260808 22:57:36.055895 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:57:36.056084 1872 external_mini_cluster.cc:1428] Setting key 34653409ebec1db4b170226d93f6674c
I20260808 22:57:36.056475 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap starting.
I20260808 22:57:36.057264 16816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:36.057586 16816 log.cc:824] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:36.058476 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: No bootstrap required, opened a new log
I20260808 22:57:36.060835 16816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.061038 16816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:36.061064 16816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Initialized, Role: FOLLOWER
I20260808 22:57:36.061216 16816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.061297 16816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:36.061342 16816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:36.061407 16816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:36.062335 16816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.062474 16816 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd0102b3e3246b79e6d4db3d64186e0; no voters:
I20260808 22:57:36.062683 16816 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:57:36.062748 16821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:36.062947 16821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 LEADER]: Becoming Leader. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Running, Role: LEADER
I20260808 22:57:36.063030 16816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:36.063089 16821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.063634 16823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd0102b3e3246b79e6d4db3d64186e0. Latest consensus state: current_term: 1 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:36.063781 16823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:36.064393 16830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:36.064965 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:36.065062 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:36.065070 16830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:36.067385 16830 catalog_manager.cc:1347] Generated new cluster ID: 4312cd12402b484abde8350f8f15faeb
I20260808 22:57:36.067440 16830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:36.068840 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.056660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.1.212.62:45295)
0808 22:57:36.056895 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.056914 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.056953 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:57:36.057509 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.057518 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.057577 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.057743 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:36.058619 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.059596 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.060876 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.061557 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.061650 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.064872 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:36.064877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:36.064877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:36.064893 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:36.067099 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.067571 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.067575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.067576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.067680 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.067894 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.067895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.067895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.068158 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:36.068330 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.068449 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.068602 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":145,"threads_started":1}
I20260808 22:57:36.089787 16830 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:57:36.090360 16830 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:36.096669 16830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Generated new TSK 0
I20260808 22:57:36.096946 16830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:57:36.132413 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:36.237962 16842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.238201 16842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.238229 16842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.239568 16842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.239620 16842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.239650 16842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:36.241318 16842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.241600 16842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.241864 16842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:36.247375 16848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.247375 16847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.247593 16842 server_base.cc:1034] running on GCE node
W20260808 22:57:36.247413 16850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.247843 16842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:36.248158 16842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:36.249321 16842 hybrid_clock.cc:648] HybridClock initialized: now 1786229856249310 us; error 36 us; skew 500 ppm
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:36.252090 16842 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:36.252581 16842 webserver.cc:460] Webserver started at http://127.1.212.1:42647/ using document root <none> and password file <none>
I20260808 22:57:36.252872 16842 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:36.252950 16842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:36.253111 16842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:36.254276 16842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "5b507642608f49a58f791a5489604ce1"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "3ee6fe39a0ff5a41de5bf9297b214c3f"
server_key_iv: "f2077deb934ffe506c992030d437d833"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.254675 16842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "5b507642608f49a58f791a5489604ce1"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "3ee6fe39a0ff5a41de5bf9297b214c3f"
server_key_iv: "f2077deb934ffe506c992030d437d833"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.256166 16842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:57:36.257129 16857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.257318 16842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:36.257438 16842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "5b507642608f49a58f791a5489604ce1"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "3ee6fe39a0ff5a41de5bf9297b214c3f"
server_key_iv: "f2077deb934ffe506c992030d437d833"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.257529 16842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:36.276252 16842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:36.278771 16842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:36.278972 16842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:36.279254 16842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:36.279618 16842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:36.279671 16842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.279711 16842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:36.279753 16842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.286088 16842 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:33717
I20260808 22:57:36.286144 16975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:33717 every 8 connection(s)
I20260808 22:57:36.286506 16842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:36.290004 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16842
I20260808 22:57:36.290128 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:57:36.290321 1872 external_mini_cluster.cc:1428] Setting key 14ccd4138ad5706bf471d303510b6615
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:36.298142 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.287270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:41189 (local address 127.1.212.62:45295)
0808 22:57:36.287360 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.287363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.287568 (+ 205us) server_negotiation.cc:407] Connection header received
0808 22:57:36.288071 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.288072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.288084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.288139 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:36.289168 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.289696 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.291177 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.291382 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.291449 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.294430 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:36.294433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:36.294433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:36.294452 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:36.296401 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.296948 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.296950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.296950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.297030 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.297298 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.297299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.297300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.297499 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:36.297578 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.297865 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.297989 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:57:36.298256 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.287236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:41189)
0808 22:57:36.287433 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.287474 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.287975 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.288197 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.288200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.288850 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:36.289070 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.289076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.289806 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.289808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.291098 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.291103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.291479 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.291480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.291635 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.292215 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:36.292228 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:36.294310 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:36.296521 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.296532 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.296535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.296868 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.297101 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.297102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.297102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.297220 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.297570 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:36.297575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:36.297715 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.297799 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.298005 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260808 22:57:36.298884 16976 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:36.299003 16976 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:36.299201 16976 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:36.299857 16778 ts_manager.cc:194] Registered new tserver with Master: 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717)
I20260808 22:57:36.300869 16778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:41189
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:57:36.321652 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:36.409651 16981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.409940 16981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.409986 16981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.411273 16981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.411358 16981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.411404 16981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:36.413102 16981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.413420 16981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.413679 16981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:36.419344 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.419348 16989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.419342 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.419368 16981 server_base.cc:1034] running on GCE node
I20260808 22:57:36.419863 16981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:36.420104 16981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:36.421247 16981 hybrid_clock.cc:648] HybridClock initialized: now 1786229856421261 us; error 72 us; skew 500 ppm
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:36.424268 16981 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:36.424870 16981 webserver.cc:460] Webserver started at http://127.1.212.2:45837/ using document root <none> and password file <none>
I20260808 22:57:36.425127 16981 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:36.425176 16981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:36.425348 16981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:36.426529 16981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "2324c26d4b86b37c80f9b255baa71a81"
server_key_iv: "ab2c12aa17458fd19b1967a3d8d8604d"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.426939 16981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "2324c26d4b86b37c80f9b255baa71a81"
server_key_iv: "ab2c12aa17458fd19b1967a3d8d8604d"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.428563 16981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:57:36.429478 16996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.429677 16981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:36.429780 16981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "2324c26d4b86b37c80f9b255baa71a81"
server_key_iv: "ab2c12aa17458fd19b1967a3d8d8604d"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.429859 16981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:36.449620 16981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:36.452013 16981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:36.452178 16981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:36.452441 16981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:36.452837 16981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:36.452890 16981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.452976 16981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:36.453006 16981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.459632 16981 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:38439
I20260808 22:57:36.459743 17114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:38439 every 8 connection(s)
I20260808 22:57:36.460121 16981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:36.468602 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 16981
I20260808 22:57:36.468731 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:57:36.468948 1872 external_mini_cluster.cc:1428] Setting key 090ee84761ac9956aad3987f908d30ab
I20260808 22:57:36.470606 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.461045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:39981 (local address 127.1.212.62:45295)
0808 22:57:36.461110 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.461113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.461335 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:57:36.461914 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.461916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.461931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.461994 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:36.463092 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.463659 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.464894 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.465116 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.465203 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.467512 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:36.467514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:36.467515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:36.467524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:36.468920 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.469478 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.469479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.469479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.469548 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.469850 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.469851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.469852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.469989 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:36.470069 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.470396 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.470504 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:57:36.470820 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.461036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:39981)
0808 22:57:36.461202 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.461243 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.461796 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.462075 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.462079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.462716 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:36.462979 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.462985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.463774 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.463775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.464794 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.464797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.465223 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.465226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.465364 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.465920 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:36.465929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:36.467402 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:36.469009 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.469020 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.469023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.469400 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.469624 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.469625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.469625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.469788 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.470075 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:36.470092 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:36.470249 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.470365 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.470552 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260808 22:57:36.472085 17116 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:36.472224 17116 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:36.472448 17116 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:36.472954 16778 ts_manager.cc:194] Registered new tserver with Master: ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439)
I20260808 22:57:36.473502 16778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:39981
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:57:36.499887 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:36.589915 17122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.590178 17122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.590206 17122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.591528 17122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.591588 17122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.591619 17122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:36.593374 17122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.593669 17122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.593937 17122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:36.599762 17127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.599689 17128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.599972 17122 server_base.cc:1034] running on GCE node
W20260808 22:57:36.599684 17130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.600244 17122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:36.600539 17122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:36.601703 17122 hybrid_clock.cc:648] HybridClock initialized: now 1786229856601693 us; error 33 us; skew 500 ppm
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:36.604617 17122 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:36.605254 17122 webserver.cc:460] Webserver started at http://127.1.212.3:35841/ using document root <none> and password file <none>
I20260808 22:57:36.605528 17122 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:36.605604 17122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:36.605759 17122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:36.606932 17122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "c8fd095566cd4133b2f9338226981784"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "5d763eb18e22d55f0af5a3a2ccb6cca2"
server_key_iv: "379abcde534d50a983198ea2e2c40b73"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.607348 17122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "c8fd095566cd4133b2f9338226981784"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "5d763eb18e22d55f0af5a3a2ccb6cca2"
server_key_iv: "379abcde534d50a983198ea2e2c40b73"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.608896 17122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:57:36.609783 17137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.609985 17122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:36.610091 17122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "c8fd095566cd4133b2f9338226981784"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "5d763eb18e22d55f0af5a3a2ccb6cca2"
server_key_iv: "379abcde534d50a983198ea2e2c40b73"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.610176 17122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:36.621436 17122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:36.624003 17122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:36.624194 17122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:36.624470 17122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:36.624877 17122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:36.624930 17122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.624975 17122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:36.625023 17122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.631633 17122 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:34743
I20260808 22:57:36.631685 17255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:34743 every 8 connection(s)
I20260808 22:57:36.632076 17122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:57:36.636734 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17122
I20260808 22:57:36.636888 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:57:36.637080 1872 external_mini_cluster.cc:1428] Setting key 775c149ba408ff7520df8988e69ce688
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:36.643605 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.633398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:54933 (local address 127.1.212.62:45295)
0808 22:57:36.633489 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.633491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.633505 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:57:36.634125 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.634126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.634139 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.634202 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:36.635368 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.635990 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.637643 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.637938 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.638023 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.640559 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:36.640561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:36.640561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:36.640570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:36.642146 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.642638 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.642639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.642639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.642698 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.642928 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.642929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.642929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.643061 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:36.643114 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.643442 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.643535 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:57:36.643787 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.633007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:54933)
0808 22:57:36.633239 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.633294 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.633989 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.634260 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.634264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.634996 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:36.635239 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.635245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.636117 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.636118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.637532 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.637536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.638036 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.638038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.638215 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.638772 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:36.638783 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:36.640423 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:36.642250 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.642262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.642264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.642551 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.642757 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:36.642757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:36.642758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:36.642874 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:36.643118 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:36.643123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:36.643265 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.643363 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.643511 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":121,"threads_started":1}
I20260808 22:57:36.644356 17256 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:36.644462 17256 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:36.644671 17256 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:36.645157 16778 ts_manager.cc:194] Registered new tserver with Master: c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
I20260808 22:57:36.645527 16778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:54933
I20260808 22:57:36.649580 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:57:36.657750 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.650974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.1.212.62:45295)
0808 22:57:36.651028 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.651030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.651181 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:57:36.651213 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.651214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.651225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.651297 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:36.651881 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.652511 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.653327 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.653532 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.653590 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.654879 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:36.654881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:36.654881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:36.654889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:36.656444 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.656914 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.656916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.656916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.656976 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:36.657154 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:36.657154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:36.657155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:36.657313 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:36.657393 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.657597 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.657679 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:57:36.659555 16778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:57:36.661448 16778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:57:36.668978 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:34743 (local address 127.0.0.1:35770)
0808 22:57:36.664797 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.664832 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.665069 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.665350 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.665354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.665356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:36.665445 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.665448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666469 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.666470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.667722 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.667726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668564 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.668668 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.668684 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.668813 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.668883 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"spinlock_wait_cycles":896,"thread_start_us":116,"threads_started":1}
I20260808 22:57:36.668991 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.1.212.3:34743)
0808 22:57:36.664936 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.664941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.665054 (+ 113us) server_negotiation.cc:407] Connection header received
0808 22:57:36.665219 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.665222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.665241 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.665327 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:36.665525 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666338 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.667830 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668444 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.668631 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.668855 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.668902 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":123,"threads_started":1}
I20260808 22:57:36.669353 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43590 (local address 127.1.212.2:38439)
0808 22:57:36.665206 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.665211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.665221 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:57:36.665263 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.665265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.665281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.665377 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:36.665583 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666770 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668010 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668750 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668856 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.668918 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.669221 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.669272 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
I20260808 22:57:36.669497 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58662 (local address 127.1.212.1:33717)
0808 22:57:36.665212 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.665216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.665225 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:36.665266 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.665269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.665283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.665357 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:36.665700 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666805 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668054 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668762 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668856 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.668919 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.669372 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.669428 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":95,"threads_started":1}
I20260808 22:57:36.669275 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:33717 (local address 127.0.0.1:58662)
0808 22:57:36.664749 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.664832 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.664904 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.665521 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.665523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.665525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:36.665599 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.665601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666914 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.666915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.667940 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.667942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668962 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.669066 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.669069 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.669126 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.669180 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":32,"thread_start_us":74,"threads_started":1}
I20260808 22:57:36.669296 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.664574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38439 (local address 127.0.0.1:43590)
0808 22:57:36.665062 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.665071 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.665116 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.665463 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.665463 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.665465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:36.665526 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.665527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.666998 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.666999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.667910 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.667913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.668960 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.668961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.669061 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.669066 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.669184 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.669219 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260808 22:57:36.670452 17186 tablet_service.cc:1467] Processing CreateTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (DEFAULT_TABLE table=test-table [id=3aed50d674714c81ab7aaccbf996ee3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:36.670574 16907 tablet_service.cc:1467] Processing CreateTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (DEFAULT_TABLE table=test-table [id=3aed50d674714c81ab7aaccbf996ee3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:36.670655 17046 tablet_service.cc:1467] Processing CreateTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (DEFAULT_TABLE table=test-table [id=3aed50d674714c81ab7aaccbf996ee3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:36.670912 16907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c95632d7ca4deab221ef5a1b4f8cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:36.670902 17186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c95632d7ca4deab221ef5a1b4f8cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:36.670948 17046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c95632d7ca4deab221ef5a1b4f8cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:36.673935 17273 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Bootstrap starting.
I20260808 22:57:36.674638 17275 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Bootstrap starting.
I20260808 22:57:36.674721 17273 tablet_bootstrap.cc:654] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:36.675243 17275 tablet_bootstrap.cc:654] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:36.675239 17273 log.cc:824] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:36.675894 17275 log.cc:824] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:36.676254 17273 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: No bootstrap required, opened a new log
I20260808 22:57:36.676312 17274 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Bootstrap starting.
I20260808 22:57:36.676376 17273 ts_tablet_manager.cc:1397] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:36.677075 17274 tablet_bootstrap.cc:654] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:36.677230 17275 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: No bootstrap required, opened a new log
I20260808 22:57:36.677320 17275 ts_tablet_manager.cc:1397] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:36.677743 17274 log.cc:824] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:36.678588 17273 raft_consensus.cc:348] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.678716 17274 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: No bootstrap required, opened a new log
I20260808 22:57:36.678783 17273 raft_consensus.cc:374] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:36.678817 17274 ts_tablet_manager.cc:1397] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:36.678856 17273 raft_consensus.cc:729] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b507642608f49a58f791a5489604ce1, State: Initialized, Role: FOLLOWER
I20260808 22:57:36.679049 17273 consensus_queue.cc:260] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.679487 16976 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
I20260808 22:57:36.679519 17273 ts_tablet_manager.cc:1428] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:36.679744 17275 raft_consensus.cc:348] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.679951 17275 raft_consensus.cc:374] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:36.680053 17275 raft_consensus.cc:729] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fd095566cd4133b2f9338226981784, State: Initialized, Role: FOLLOWER
I20260808 22:57:36.680194 17275 consensus_queue.cc:260] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.680533 17275 ts_tablet_manager.cc:1428] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:57:36.680590 17256 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
I20260808 22:57:36.681370 17274 raft_consensus.cc:348] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.681608 17274 raft_consensus.cc:374] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:36.681661 17274 raft_consensus.cc:729] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8c98fc1a114649b56dfc9febc17d8d, State: Initialized, Role: FOLLOWER
I20260808 22:57:36.681826 17274 consensus_queue.cc:260] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.682118 17274 ts_tablet_manager.cc:1428] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:36.682163 17116 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
I20260808 22:57:36.682512 17280 raft_consensus.cc:482] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:36.682607 17280 raft_consensus.cc:504] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.682936 17280 leader_election.cc:288] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717), ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439)
I20260808 22:57:36.685881 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.683058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:33717 (local address 127.1.212.3:58831)
0808 22:57:36.683168 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.683181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.683223 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.683368 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.683369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.683370 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:36.683463 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.683465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.684144 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.684145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.685003 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.685006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.685603 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.685699 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.685704 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.685762 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.685810 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":1}
I20260808 22:57:36.685901 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.683163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:58831 (local address 127.1.212.1:33717)
0808 22:57:36.683213 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.683215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.683224 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:36.683287 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.683288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.683303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.683356 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:36.683531 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.684057 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685101 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.685512 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685569 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.685579 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.685777 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.685817 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:57:36.686080 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.683173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:34831 (local address 127.1.212.2:38439)
0808 22:57:36.683215 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.683217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.683341 (+ 124us) server_negotiation.cc:407] Connection header received
0808 22:57:36.683401 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.683401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.683411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.683466 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:36.683622 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.684071 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685096 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.685662 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.685762 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.685929 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.685978 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:57:36.686434 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.683082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38439 (local address 127.1.212.3:34831)
0808 22:57:36.683281 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.683291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:36.683353 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:36.683480 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:36.683480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:36.683481 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:36.683566 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.683568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.684168 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.684169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.685009 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:36.685012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.685773 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.685774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:36.685848 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.685853 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:36.685912 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 22:57:36.685956 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
I20260808 22:57:36.686427 16927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c95632d7ca4deab221ef5a1b4f8cb0" candidate_uuid: "c8fd095566cd4133b2f9338226981784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b507642608f49a58f791a5489604ce1" is_pre_election: true
I20260808 22:57:36.686602 16927 raft_consensus.cc:2454] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fd095566cd4133b2f9338226981784 in term 0.
I20260808 22:57:36.686764 17067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c95632d7ca4deab221ef5a1b4f8cb0" candidate_uuid: "c8fd095566cd4133b2f9338226981784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" is_pre_election: true
I20260808 22:57:36.686821 17140 leader_election.cc:302] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b507642608f49a58f791a5489604ce1, c8fd095566cd4133b2f9338226981784; no voters:
I20260808 22:57:36.686887 17067 raft_consensus.cc:2454] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fd095566cd4133b2f9338226981784 in term 0.
I20260808 22:57:36.686949 17280 raft_consensus.cc:2781] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:57:36.687007 17280 raft_consensus.cc:482] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:57:36.687049 17280 raft_consensus.cc:3037] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:36.687800 17280 raft_consensus.cc:504] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.687953 17280 leader_election.cc:288] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [CANDIDATE]: Term 1 election: Requested vote from peers 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717), ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439)
I20260808 22:57:36.688184 16927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c95632d7ca4deab221ef5a1b4f8cb0" candidate_uuid: "c8fd095566cd4133b2f9338226981784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b507642608f49a58f791a5489604ce1"
I20260808 22:57:36.688248 17067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c95632d7ca4deab221ef5a1b4f8cb0" candidate_uuid: "c8fd095566cd4133b2f9338226981784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
I20260808 22:57:36.688274 16927 raft_consensus.cc:3037] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:36.688354 17067 raft_consensus.cc:3037] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:36.689179 16927 raft_consensus.cc:2454] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fd095566cd4133b2f9338226981784 in term 1.
I20260808 22:57:36.689254 17067 raft_consensus.cc:2454] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fd095566cd4133b2f9338226981784 in term 1.
I20260808 22:57:36.689379 17140 leader_election.cc:302] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b507642608f49a58f791a5489604ce1, c8fd095566cd4133b2f9338226981784; no voters:
I20260808 22:57:36.689487 17280 raft_consensus.cc:2781] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:36.689682 17280 raft_consensus.cc:686] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 1 LEADER]: Becoming Leader. State: Replica: c8fd095566cd4133b2f9338226981784, State: Running, Role: LEADER
I20260808 22:57:36.689791 17280 consensus_queue.cc:237] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:36.690747 16777 catalog_manager.cc:5495] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8fd095566cd4133b2f9338226981784 (127.1.212.3). New cstate: current_term: 1 leader_uuid: "c8fd095566cd4133b2f9338226981784" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } health_report { overall_health: UNKNOWN } } }
I20260808 22:57:36.697418 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.694953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.1.212.3:34743)
0808 22:57:36.695015 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:36.695022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:36.695034 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:57:36.695071 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:36.695072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:36.695091 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:36.695163 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:57:36.695268 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.695762 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.696668 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:36.696930 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:36.696987 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:36.697135 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:57:36.697245 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:57:36.697304 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:36.697316 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 22:57:36.697337 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:57:36.699818 17067 raft_consensus.cc:1261] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Refusing update from remote peer c8fd095566cd4133b2f9338226981784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:36.699818 16927 raft_consensus.cc:1261] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Refusing update from remote peer c8fd095566cd4133b2f9338226981784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:36.700299 17283 consensus_queue.cc:1035] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:36.700481 17280 consensus_queue.cc:1035] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:36.703692 17286 mvcc.cc:205] Tried to move back new op lower bound from 7316397493038739456 to 7316397493002158080. Current Snapshot: MvccSnapshot[applied={T|T < 7316397493038739456}]
I20260808 22:57:36.704849 17287 mvcc.cc:205] Tried to move back new op lower bound from 7316397493038739456 to 7316397493002158080. Current Snapshot: MvccSnapshot[applied={T|T < 7316397493038739456}]
I20260808 22:57:36.706354 16778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:36.706489 16778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:36.707322 16778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Sending DeleteTablet for 3 replicas of tablet 26c95632d7ca4deab221ef5a1b4f8cb0
I20260808 22:57:36.707729 17186 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:35770
I20260808 22:57:36.707778 17046 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43590
I20260808 22:57:36.707772 16907 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:58662
I20260808 22:57:36.707867 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16842
I20260808 22:57:36.714206 17297 tablet_replica.cc:330] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: stopping tablet replica
I20260808 22:57:36.714421 17297 raft_consensus.cc:2229] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:57:36.714578 17297 raft_consensus.cc:2258] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:36.714779 17298 tablet_replica.cc:330] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: stopping tablet replica
I20260808 22:57:36.714910 17298 raft_consensus.cc:2229] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:36.715049 17298 raft_consensus.cc:2258] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:36.715575 17297 ts_tablet_manager.cc:1905] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:36.717528 17298 ts_tablet_manager.cc:1905] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:36.718065 17297 ts_tablet_manager.cc:1918] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:36.718262 17297 log.cc:1196] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/26c95632d7ca4deab221ef5a1b4f8cb0
I20260808 22:57:36.718842 17297 ts_tablet_manager.cc:1939] T 26c95632d7ca4deab221ef5a1b4f8cb0 P c8fd095566cd4133b2f9338226981784: Deleting consensus metadata
I20260808 22:57:36.719611 17298 ts_tablet_manager.cc:1918] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:36.719712 17298 log.cc:1196] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/26c95632d7ca4deab221ef5a1b4f8cb0
I20260808 22:57:36.720073 17298 ts_tablet_manager.cc:1939] T 26c95632d7ca4deab221ef5a1b4f8cb0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting consensus metadata
I20260808 22:57:36.720422 16761 catalog_manager.cc:4841] TS c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (table test-table [id=3aed50d674714c81ab7aaccbf996ee3f]) successfully deleted
I20260808 22:57:36.720588 16761 catalog_manager.cc:4841] TS ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (table test-table [id=3aed50d674714c81ab7aaccbf996ee3f]) successfully deleted
W20260808 22:57:36.722925 16763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:33717 (error 108)
I20260808 22:57:36.723313 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16981
I20260808 22:57:36.723742 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:36.723581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:33717 (local address 127.0.0.1:58674)
0808 22:57:36.723652 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:36.723697 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 22:57:36.724367 16763 catalog_manager.cc:4584] TS 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26c95632d7ca4deab221ef5a1b4f8cb0: Network error: Client connection negotiation failed: client connection to 127.1.212.1:33717: connect: Connection refused (error 111)
I20260808 22:57:36.736016 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17122
I20260808 22:57:36.743145 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 16742
I20260808 22:57:36.751183 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--webserver_interface=127.1.212.62
--webserver_port=41771
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:45295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:36.841905 17301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.842197 17301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.842239 17301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.843497 17301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:36.843570 17301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.843598 17301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.843627 17301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:36.843653 17301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:36.845337 17301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:45295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=41771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.845628 17301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.845872 17301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:36.857996 17312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.858249 17301 server_base.cc:1034] running on GCE node
W20260808 22:57:36.857996 17307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:36.858033 17308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:36.858719 17301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:36.859092 17301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:36.860267 17301 hybrid_clock.cc:648] HybridClock initialized: now 1786229856860252 us; error 48 us; skew 500 ppm
Aug 08 22:57:36 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229856, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:36.863144 17301 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:36.863692 17301 webserver.cc:460] Webserver started at http://127.1.212.62:41771/ using document root <none> and password file <none>
I20260808 22:57:36.863951 17301 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:36.864042 17301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:36.865615 17301 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:36.866319 17319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:36.866498 17301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:57:36.866612 17301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "0dd0102b3e3246b79e6d4db3d64186e0"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "1e4f1e23c1c6379e9b5a0847b9dc4d66"
server_key_iv: "96e6dc4fce56723229a185536e250860"
server_key_version: "encryptionkey@0"
I20260808 22:57:36.867066 17301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:36.881428 17301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:36.883579 17301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:36.883764 17301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:36.888104 17301 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:45295
I20260808 22:57:36.888141 17373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:45295 every 8 connection(s)
I20260808 22:57:36.888576 17301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:36.890406 17374 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:57:36.891065 17374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap starting.
I20260808 22:57:36.898067 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17301
I20260808 22:57:36.898451 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:33717
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=42647
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:36.900712 17374 log.cc:824] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:36.903009 17374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:57:36.903380 17374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap complete.
I20260808 22:57:36.905922 17374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.906169 17374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Initialized, Role: FOLLOWER
I20260808 22:57:36.906342 17374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.906406 17374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:36.906478 17374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:36.906531 17374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:57:36.907652 17374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.907749 17374 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd0102b3e3246b79e6d4db3d64186e0; no voters:
I20260808 22:57:36.907954 17374 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:57:36.907997 17378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:57:36.908166 17378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 LEADER]: Becoming Leader. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Running, Role: LEADER
I20260808 22:57:36.908259 17374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:36.908330 17378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:36.908716 17379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:36.908857 17379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:36.908710 17378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd0102b3e3246b79e6d4db3d64186e0. Latest consensus state: current_term: 2 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:36.909296 17378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:36.909334 17388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:36.910356 17388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:36.910564 17388 catalog_manager.cc:1259] Loaded cluster ID: 4312cd12402b484abde8350f8f15faeb
I20260808 22:57:36.910768 17388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:36.911038 17388 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 22:57:36.916189 17388 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 22:57:36.919214 17388 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:36.919497 17388 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Loaded TSK: 0
I20260808 22:57:36.919761 17388 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:57:36.991752 17376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:36.992048 17376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:36.992097 17376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:36.993501 17376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:36.993579 17376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:36.993630 17376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:36.995321 17376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:33717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=42647
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:36.995649 17376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:36.995903 17376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:37.002466 17402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.002523 17376 server_base.cc:1034] running on GCE node
W20260808 22:57:37.002602 17405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.002682 17403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.003042 17376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:37.003320 17376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:37.004531 17376 hybrid_clock.cc:648] HybridClock initialized: now 1786229857004508 us; error 40 us; skew 500 ppm
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:37.007529 17376 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:37.008071 17376 webserver.cc:460] Webserver started at http://127.1.212.1:42647/ using document root <none> and password file <none>
I20260808 22:57:37.008320 17376 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:37.008414 17376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:37.010094 17376 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.010861 17412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.011046 17376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.011153 17376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "5b507642608f49a58f791a5489604ce1"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "3ee6fe39a0ff5a41de5bf9297b214c3f"
server_key_iv: "f2077deb934ffe506c992030d437d833"
server_key_version: "encryptionkey@0"
I20260808 22:57:37.011550 17376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:37.045819 17376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:37.047961 17376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:37.048141 17376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:37.048404 17376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:37.048928 17419 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:57:37.050633 17376 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:57:37.050698 17376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 22:57:37.050747 17376 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:57:37.051445 17376 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:57:37.051497 17376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:37.051582 17419 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Bootstrap starting.
I20260808 22:57:37.058828 17376 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:33717
I20260808 22:57:37.058917 17530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:33717 every 8 connection(s)
I20260808 22:57:37.059306 17376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:37.063350 17419 log.cc:824] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:37.065793 17419 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:57:37.065994 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17376
I20260808 22:57:37.066180 17419 tablet_bootstrap.cc:492] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Bootstrap complete.
I20260808 22:57:37.066311 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:38439
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=45837
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:37.067323 17419 ts_tablet_manager.cc:1397] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:37.068959 17419 raft_consensus.cc:348] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:37.069381 17419 raft_consensus.cc:729] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b507642608f49a58f791a5489604ce1, State: Initialized, Role: FOLLOWER
I20260808 22:57:37.069582 17419 consensus_queue.cc:260] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } }
I20260808 22:57:37.069957 17419 ts_tablet_manager.cc:1428] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260808 22:57:37.072365 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.060113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:52197)
0808 22:57:37.060372 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.060446 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.061006 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.061553 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.061557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.061829 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:37.062034 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.062039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.063810 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.063811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.064721 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.064724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.065425 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.065426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.065579 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.066197 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:37.066209 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:37.068217 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:37.070686 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.070692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.070694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.070930 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.071250 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.071251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.071251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.071351 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.071844 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:37.071849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:37.071975 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.072081 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.072218 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":148,"threads_started":1}
I20260808 22:57:37.072839 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.060192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:52197 (local address 127.1.212.62:45295)
0808 22:57:37.060556 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.060568 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.060627 (+ 59us) server_negotiation.cc:407] Connection header received
0808 22:57:37.061263 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.061275 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.061346 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.061516 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:37.062177 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.063668 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.064873 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.065293 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.065431 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.068343 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:37.068349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:37.068349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:37.068364 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:37.070601 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.071028 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.071033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.071034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.071169 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.071417 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.071419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.071419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.071768 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:37.071867 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.072179 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.072349 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":98,"threads_started":1}
I20260808 22:57:37.073549 17531 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:37.073669 17531 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:37.073894 17531 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:37.074824 17336 ts_manager.cc:194] Registered new tserver with Master: 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717)
I20260808 22:57:37.075949 17336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:52197
I20260808 22:57:37.077135 17531 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
I20260808 22:57:37.080102 17539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.075508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:33717 (local address 127.0.0.1:58682)
0808 22:57:37.075700 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.075749 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.075827 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.077123 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.077128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.077130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:37.077211 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.077213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.078227 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.078229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.079069 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.079072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.079754 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.079755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.079854 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.079867 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.079959 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.080026 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260808 22:57:37.080101 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.075592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.1.212.1:33717)
0808 22:57:37.076928 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.076933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.076959 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:57:37.076993 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.076995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.077008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.077097 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:37.077294 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.078138 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.079167 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.079651 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.079749 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.079813 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.079978 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.080023 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":86,"threads_started":1}
I20260808 22:57:37.080545 17463 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c95632d7ca4deab221ef5a1b4f8cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:36 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:58682
I20260808 22:57:37.081166 17542 tablet_replica.cc:330] stopping tablet replica
I20260808 22:57:37.081288 17542 raft_consensus.cc:2229] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:37.081362 17542 raft_consensus.cc:2258] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:37.081707 17542 ts_tablet_manager.cc:1905] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:37.084481 17542 ts_tablet_manager.cc:1918] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:37.084583 17542 log.cc:1196] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/26c95632d7ca4deab221ef5a1b4f8cb0
I20260808 22:57:37.084856 17542 ts_tablet_manager.cc:1939] T 26c95632d7ca4deab221ef5a1b4f8cb0 P 5b507642608f49a58f791a5489604ce1: Deleting consensus metadata
I20260808 22:57:37.085494 17322 catalog_manager.cc:4841] TS 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717): tablet 26c95632d7ca4deab221ef5a1b4f8cb0 (table test-table [id=3aed50d674714c81ab7aaccbf996ee3f]) successfully deleted
W20260808 22:57:37.159162 17537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:37.159407 17537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:37.159435 17537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:37.160694 17537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:37.160749 17537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:37.160830 17537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:37.162433 17537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:38439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=45837
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:37.162706 17537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:37.162964 17537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:37.168380 17550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.168380 17548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.168547 17537 server_base.cc:1034] running on GCE node
W20260808 22:57:37.168421 17547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.169044 17537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:37.169272 17537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:37.170418 17537 hybrid_clock.cc:648] HybridClock initialized: now 1786229857170406 us; error 34 us; skew 500 ppm
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:37.173246 17537 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:37.173732 17537 webserver.cc:460] Webserver started at http://127.1.212.2:45837/ using document root <none> and password file <none>
I20260808 22:57:37.173974 17537 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:37.174058 17537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:37.175545 17537 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.176193 17557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.176370 17537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.176483 17537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "2324c26d4b86b37c80f9b255baa71a81"
server_key_iv: "ab2c12aa17458fd19b1967a3d8d8604d"
server_key_version: "encryptionkey@0"
I20260808 22:57:37.176919 17537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:37.193678 17537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:37.195837 17537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:37.196014 17537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:37.196278 17537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:37.196616 17537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:37.196671 17537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.196713 17537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:37.196744 17537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.203001 17537 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:38439
I20260808 22:57:37.203049 17677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:38439 every 8 connection(s)
I20260808 22:57:37.203446 17537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:57:37.203881 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17537
I20260808 22:57:37.204147 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:34743
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=35841
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:37.214097 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.204264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:54607 (local address 127.1.212.62:45295)
0808 22:57:37.204477 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.204480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.204621 (+ 141us) server_negotiation.cc:407] Connection header received
0808 22:57:37.205198 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.205199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.205214 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.205297 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:37.206318 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.206836 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.207889 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.208165 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.208232 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.210948 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:37.210950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:37.210950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:37.210958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:37.212440 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.212976 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.212978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.212978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.213047 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.213321 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.213322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.213322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.213450 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:37.213523 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.213857 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.213987 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260808 22:57:37.214314 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.204276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:54607)
0808 22:57:37.204464 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.204520 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.205099 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.205372 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.205376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.206001 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:37.206203 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.206208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.206947 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.206948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.207781 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.207784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.208242 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.208244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.208378 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.208975 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:37.208986 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:37.210828 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:37.212545 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.212555 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.212558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.212885 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.213121 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.213122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.213122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.213248 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.213533 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:37.213542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:37.213696 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.213802 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.213992 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260808 22:57:37.214876 17678 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:37.215062 17678 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:37.215332 17678 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:37.215808 17336 ts_manager.cc:194] Registered new tserver with Master: ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439)
I20260808 22:57:37.216415 17336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:54607
W20260808 22:57:37.293577 17681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:37.293814 17681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:37.293843 17681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:37.295020 17681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:37.295073 17681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:37.295101 17681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:37.296610 17681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:34743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=35841
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:37.296903 17681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:37.297142 17681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:37.302575 17687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.302608 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.302575 17686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.302789 17681 server_base.cc:1034] running on GCE node
I20260808 22:57:37.303025 17681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:37.303339 17681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:37.304515 17681 hybrid_clock.cc:648] HybridClock initialized: now 1786229857304507 us; error 38 us; skew 500 ppm
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:37.307273 17681 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:37.307735 17681 webserver.cc:460] Webserver started at http://127.1.212.3:35841/ using document root <none> and password file <none>
I20260808 22:57:37.308001 17681 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:37.308094 17681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:37.309719 17681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.310411 17697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.310602 17681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.310696 17681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "c8fd095566cd4133b2f9338226981784"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "5d763eb18e22d55f0af5a3a2ccb6cca2"
server_key_iv: "379abcde534d50a983198ea2e2c40b73"
server_key_version: "encryptionkey@0"
I20260808 22:57:37.311116 17681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:37.326486 17681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:37.329129 17681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:37.329308 17681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:37.329579 17681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:37.329950 17681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:37.330022 17681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.330065 17681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:37.330092 17681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.336486 17681 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:34743
I20260808 22:57:37.336541 17817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:34743 every 8 connection(s)
I20260808 22:57:37.336977 17681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:57:37.340821 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17681
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:37.347988 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.337824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:38585 (local address 127.1.212.62:45295)
0808 22:57:37.337908 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.337911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.338123 (+ 212us) server_negotiation.cc:407] Connection header received
0808 22:57:37.338644 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.338646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.338659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.338734 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:37.339744 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.340315 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.341592 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.341868 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.341922 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.344629 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:37.344630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:37.344630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:37.344637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:37.346416 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.346823 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.346824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.346825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.346882 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.347189 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.347190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.347191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.347318 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:37.347387 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.347723 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.347889 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:57:37.348150 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.337810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:38585)
0808 22:57:37.337979 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.338031 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.338532 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.338794 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.338797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.339407 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:37.339624 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.339630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.340420 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.340420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.341508 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.341512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.341963 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.341964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.342184 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.342822 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:37.342833 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:37.344543 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:37.346504 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.346514 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.346516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.346755 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.346972 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:37.346973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:37.346974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:37.347112 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:37.347392 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:37.347400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:37.347576 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.347686 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.347889 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 22:57:37.348657 17818 heartbeater.cc:347] Connected to a master server at 127.1.212.62:45295
I20260808 22:57:37.348829 17818 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:37.349080 17818 heartbeater.cc:511] Master 127.1.212.62:45295 requested a full tablet report, sending...
I20260808 22:57:37.349308 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.341305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.1.212.62:45295)
0808 22:57:37.341450 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.341452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.341561 (+ 109us) server_negotiation.cc:407] Connection header received
0808 22:57:37.341617 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.341617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.341630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.341698 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:37.342793 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.343283 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.344139 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.344452 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.344532 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.346072 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:37.346073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:37.346073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:37.346081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:37.347934 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.348371 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.348372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.348373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.348440 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:37.348645 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:37.348646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:37.348647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:37.348828 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:37.349018 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.349103 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.349209 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260808 22:57:37.349574 17336 ts_manager.cc:194] Registered new tserver with Master: c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
I20260808 22:57:37.350024 17336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:38585
I20260808 22:57:37.350147 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:57:37.353230 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.350633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.1.212.62:45295)
0808 22:57:37.350692 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.350693 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.350720 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:57:37.350815 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.350816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.350859 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.350939 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:57:37.351101 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.351580 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.352369 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.352596 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.352671 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.352883 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:57:37.352998 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:57:37.353095 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.353115 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.353143 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:57:37.353583 17336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:57:37.355357 17336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:57:37.357810 17463 tablet_service.cc:1467] Processing CreateTablet for tablet be86e3bc6a414583bc66a352ca95fee0 (DEFAULT_TABLE table=test-table [id=889ae3860d8f44d19deaf9f4282d973e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:37.358058 17463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be86e3bc6a414583bc66a352ca95fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:37.359669 17419 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Bootstrap starting.
I20260808 22:57:37.359984 17419 tablet_bootstrap.cc:654] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:37.360689 17419 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: No bootstrap required, opened a new log
I20260808 22:57:37.360759 17419 ts_tablet_manager.cc:1397] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:37.360905 17419 raft_consensus.cc:348] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.360962 17419 raft_consensus.cc:374] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:37.360997 17419 raft_consensus.cc:729] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b507642608f49a58f791a5489604ce1, State: Initialized, Role: FOLLOWER
I20260808 22:57:37.361059 17419 consensus_queue.cc:260] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.361153 17419 ts_tablet_manager.cc:1428] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.361595 17539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.357368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38439 (local address 127.0.0.1:43594)
0808 22:57:37.357413 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.357425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.357463 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.358045 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.358046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.358048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:37.358094 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.358096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.359179 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.359180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.360339 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.360342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.361355 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.361356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.361430 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.361435 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.361467 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.361520 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 22:57:37.361735 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.357468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:34743 (local address 127.0.0.1:35792)
0808 22:57:37.357679 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.357689 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.357738 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.358056 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.358057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.358060 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:37.358112 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.358115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.359360 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.359361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.360417 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.360419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.361425 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.361426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.361488 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.361491 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.361550 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.361584 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":129,"threads_started":1}
I20260808 22:57:37.361779 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.357579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.1.212.3:34743)
0808 22:57:37.357793 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.357798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.357812 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:57:37.357857 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.357859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.357877 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.357966 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:37.358201 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.359246 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.360515 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.361248 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.361326 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.361394 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.361643 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.361693 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260808 22:57:37.362284 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.357477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43594 (local address 127.1.212.2:38439)
0808 22:57:37.357864 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.357868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.357876 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:57:37.357926 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.357928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.357945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.358187 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:37.358268 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.359073 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.360446 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.361248 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.362087 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.362164 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.362188 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.362222 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":122,"threads_started":1}
I20260808 22:57:37.362589 17746 tablet_service.cc:1467] Processing CreateTablet for tablet be86e3bc6a414583bc66a352ca95fee0 (DEFAULT_TABLE table=test-table [id=889ae3860d8f44d19deaf9f4282d973e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:37.362974 17746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be86e3bc6a414583bc66a352ca95fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:37.362960 17608 tablet_service.cc:1467] Processing CreateTablet for tablet be86e3bc6a414583bc66a352ca95fee0 (DEFAULT_TABLE table=test-table [id=889ae3860d8f44d19deaf9f4282d973e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:37.363284 17608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be86e3bc6a414583bc66a352ca95fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:37.366847 17828 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Bootstrap starting.
I20260808 22:57:37.367054 17829 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Bootstrap starting.
I20260808 22:57:37.367700 17829 tablet_bootstrap.cc:654] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:37.367705 17828 tablet_bootstrap.cc:654] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:37.368376 17828 log.cc:824] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:37.368376 17829 log.cc:824] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:37.369261 17829 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: No bootstrap required, opened a new log
I20260808 22:57:37.369275 17828 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: No bootstrap required, opened a new log
I20260808 22:57:37.369323 17829 ts_tablet_manager.cc:1397] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:57:37.369359 17828 ts_tablet_manager.cc:1397] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:37.370841 17829 raft_consensus.cc:348] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.371012 17829 raft_consensus.cc:374] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:37.371062 17829 raft_consensus.cc:729] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8c98fc1a114649b56dfc9febc17d8d, State: Initialized, Role: FOLLOWER
I20260808 22:57:37.370965 17828 raft_consensus.cc:348] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.371205 17828 raft_consensus.cc:374] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:37.371199 17829 consensus_queue.cc:260] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.371263 17828 raft_consensus.cc:729] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fd095566cd4133b2f9338226981784, State: Initialized, Role: FOLLOWER
I20260808 22:57:37.371403 17828 consensus_queue.cc:260] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.371563 17829 ts_tablet_manager.cc:1428] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:57:37.371652 17678 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
I20260808 22:57:37.371768 17828 ts_tablet_manager.cc:1428] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 22:57:37.371846 17818 heartbeater.cc:503] Master 127.1.212.62:45295 was elected leader, sending a full tablet report...
W20260808 22:57:37.454138 17679 tablet.cc:2369] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:57:37.560160 17532 tablet.cc:2369] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:57:37.587584 17819 tablet.cc:2369] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:57:37.675060 17538 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:37.675324 17538 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.675712 17538 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
I20260808 22:57:37.680150 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.676142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:36457 (local address 127.1.212.2:38439)
0808 22:57:37.676231 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.676233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.676248 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:57:37.676386 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.676388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.676404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.676471 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:37.676734 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.677545 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679149 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.679748 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679831 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.679845 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.680018 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.680072 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:57:37.680150 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.676365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:33249 (local address 127.1.212.3:34743)
0808 22:57:37.676432 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.676434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.676443 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:37.676517 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.676518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.676530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.676580 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:37.676795 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.677533 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679149 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.679793 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679849 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.679860 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.680030 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.680072 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:57:37.680188 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.675992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38439 (local address 127.1.212.1:36457)
0808 22:57:37.676169 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.676182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.676265 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.676500 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.676501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.676504 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:37.676607 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.676610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.677646 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.677648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.679035 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.679040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.679846 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.679919 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.679925 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.680001 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.680060 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260808 22:57:37.680189 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.676206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:34743 (local address 127.1.212.1:33249)
0808 22:57:37.676320 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.676332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:37.676406 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:37.676614 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:37.676614 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:37.676616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:37.676677 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.676678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.677648 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.677648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.679035 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:37.679040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.679894 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.679895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:37.679949 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.679953 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:57:37.680013 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:57:37.680060 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":102,"thread_start_us":45,"threads_started":1}
I20260808 22:57:37.680697 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be86e3bc6a414583bc66a352ca95fee0" candidate_uuid: "5b507642608f49a58f791a5489604ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fd095566cd4133b2f9338226981784" is_pre_election: true
I20260808 22:57:37.680855 17629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be86e3bc6a414583bc66a352ca95fee0" candidate_uuid: "5b507642608f49a58f791a5489604ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" is_pre_election: true
I20260808 22:57:37.680922 17768 raft_consensus.cc:2454] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b507642608f49a58f791a5489604ce1 in term 0.
I20260808 22:57:37.681044 17629 raft_consensus.cc:2454] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b507642608f49a58f791a5489604ce1 in term 0.
I20260808 22:57:37.681233 17413 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b507642608f49a58f791a5489604ce1, c8fd095566cd4133b2f9338226981784; no voters:
I20260808 22:57:37.681377 17538 raft_consensus.cc:2781] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:57:37.681438 17538 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:57:37.681461 17538 raft_consensus.cc:3037] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:37.682171 17538 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.682310 17538 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 1 election: Requested vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
I20260808 22:57:37.682487 17629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be86e3bc6a414583bc66a352ca95fee0" candidate_uuid: "5b507642608f49a58f791a5489604ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
I20260808 22:57:37.682484 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be86e3bc6a414583bc66a352ca95fee0" candidate_uuid: "5b507642608f49a58f791a5489604ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fd095566cd4133b2f9338226981784"
I20260808 22:57:37.682554 17629 raft_consensus.cc:3037] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:37.682590 17768 raft_consensus.cc:3037] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:37.683284 17629 raft_consensus.cc:2454] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b507642608f49a58f791a5489604ce1 in term 1.
I20260808 22:57:37.683450 17413 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b507642608f49a58f791a5489604ce1, ce8c98fc1a114649b56dfc9febc17d8d; no voters:
I20260808 22:57:37.683450 17768 raft_consensus.cc:2454] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b507642608f49a58f791a5489604ce1 in term 1.
I20260808 22:57:37.683576 17538 raft_consensus.cc:2781] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:37.683709 17538 raft_consensus.cc:686] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 LEADER]: Becoming Leader. State: Replica: 5b507642608f49a58f791a5489604ce1, State: Running, Role: LEADER
I20260808 22:57:37.683799 17538 consensus_queue.cc:237] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:37.684589 17336 catalog_manager.cc:5495] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b507642608f49a58f791a5489604ce1 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "5b507642608f49a58f791a5489604ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } health_report { overall_health: HEALTHY } } }
I20260808 22:57:37.719049 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.715975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.1.212.1:33717)
0808 22:57:37.716132 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:37.716134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:37.716140 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:57:37.716183 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:37.716184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:37.716204 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:37.716293 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:57:37.716424 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.717392 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.718171 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:37.718381 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:37.718453 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:37.718655 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:57:37.718769 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:57:37.718858 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:37.718900 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 22:57:37.718925 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260808 22:57:37.720829 17768 raft_consensus.cc:1261] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Refusing update from remote peer 5b507642608f49a58f791a5489604ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:37.720829 17629 raft_consensus.cc:1261] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Refusing update from remote peer 5b507642608f49a58f791a5489604ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:37.721253 17538 consensus_queue.cc:1035] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:37.721414 17836 consensus_queue.cc:1035] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:37.724161 17841 mvcc.cc:205] Tried to move back new op lower bound from 7316397497221373952 to 7316397497073442816. Current Snapshot: MvccSnapshot[applied={T|T < 7316397497221373952 or (T in {7316397497221373952})}]
I20260808 22:57:37.723934 17842 mvcc.cc:205] Tried to move back new op lower bound from 7316397497221373952 to 7316397497073442816. Current Snapshot: MvccSnapshot[applied={T|T < 7316397497221373952}]
I20260808 22:57:37.725350 17843 mvcc.cc:205] Tried to move back new op lower bound from 7316397497221373952 to 7316397497073442816. Current Snapshot: MvccSnapshot[applied={T|T < 7316397497221373952 or (T in {7316397497221373952})}]
I20260808 22:57:37.725880 17336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:37.725991 17336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:37.726756 17336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Sending DeleteTablet for 3 replicas of tablet be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:37.727063 17463 tablet_service.cc:1514] Processing DeleteTablet for tablet be86e3bc6a414583bc66a352ca95fee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:37 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:58682
I20260808 22:57:37.727157 17746 tablet_service.cc:1514] Processing DeleteTablet for tablet be86e3bc6a414583bc66a352ca95fee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:37 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:35792
I20260808 22:57:37.727242 17851 tablet_replica.cc:330] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: stopping tablet replica
I20260808 22:57:37.727245 17608 tablet_service.cc:1514] Processing DeleteTablet for tablet be86e3bc6a414583bc66a352ca95fee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:37 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43594
I20260808 22:57:37.727293 17851 raft_consensus.cc:2229] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:57:37.727367 17851 raft_consensus.cc:2258] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:37.727566 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17376
I20260808 22:57:37.727751 17851 ts_tablet_manager.cc:1905] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:37.730005 17852 tablet_replica.cc:330] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: stopping tablet replica
I20260808 22:57:37.730118 17852 raft_consensus.cc:2229] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:37.730260 17852 raft_consensus.cc:2258] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:37.730494 17853 tablet_replica.cc:330] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: stopping tablet replica
I20260808 22:57:37.730624 17853 raft_consensus.cc:2229] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:37.730753 17853 raft_consensus.cc:2258] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:37.730830 17852 ts_tablet_manager.cc:1905] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:37.731281 17853 ts_tablet_manager.cc:1905] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:37.732808 17852 ts_tablet_manager.cc:1918] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:37.732903 17852 log.cc:1196] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:37.733075 17853 ts_tablet_manager.cc:1918] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:37.733144 17853 log.cc:1196] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:37.733253 17852 ts_tablet_manager.cc:1939] T be86e3bc6a414583bc66a352ca95fee0 P c8fd095566cd4133b2f9338226981784: Deleting consensus metadata
I20260808 22:57:37.733407 17853 ts_tablet_manager.cc:1939] T be86e3bc6a414583bc66a352ca95fee0 P ce8c98fc1a114649b56dfc9febc17d8d: Deleting consensus metadata
I20260808 22:57:37.733788 17320 catalog_manager.cc:4841] TS c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): tablet be86e3bc6a414583bc66a352ca95fee0 (table test-table [id=889ae3860d8f44d19deaf9f4282d973e]) successfully deleted
I20260808 22:57:37.737023 17320 catalog_manager.cc:4841] TS ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): tablet be86e3bc6a414583bc66a352ca95fee0 (table test-table [id=889ae3860d8f44d19deaf9f4282d973e]) successfully deleted
W20260808 22:57:37.738174 17322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:33717 (error 108)
I20260808 22:57:37.738735 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:37.738612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:33717 (local address 127.0.0.1:58700)
0808 22:57:37.738671 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:37.738705 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 22:57:37.738852 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17537
W20260808 22:57:37.738873 17322 catalog_manager.cc:4584] TS 5b507642608f49a58f791a5489604ce1 (127.1.212.1:33717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be86e3bc6a414583bc66a352ca95fee0: Network error: Client connection negotiation failed: client connection to 127.1.212.1:33717: connect: Connection refused (error 111)
I20260808 22:57:37.746188 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17681
I20260808 22:57:37.753167 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17301
I20260808 22:57:37.760242 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--webserver_interface=127.1.212.62
--webserver_port=41771
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:45295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:37.846809 17855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:37.847100 17855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:37.847146 17855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:37.848380 17855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:37.848453 17855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:37.848481 17855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:37.848502 17855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:37.848531 17855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:37.850435 17855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:45295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=41771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:37.850752 17855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:37.850997 17855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:37.856165 17861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.856170 17860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:37.856170 17863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:37.856665 17855 server_base.cc:1034] running on GCE node
I20260808 22:57:37.856967 17855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:37.857285 17855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:37.858467 17855 hybrid_clock.cc:648] HybridClock initialized: now 1786229857858456 us; error 42 us; skew 500 ppm
Aug 08 22:57:37 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:37.861240 17855 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:37.861792 17855 webserver.cc:460] Webserver started at http://127.1.212.62:41771/ using document root <none> and password file <none>
I20260808 22:57:37.862039 17855 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:37.862109 17855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:37.863559 17855 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 22:57:37.864282 17870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:37.864490 17855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:37.864596 17855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "0dd0102b3e3246b79e6d4db3d64186e0"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "1e4f1e23c1c6379e9b5a0847b9dc4d66"
server_key_iv: "96e6dc4fce56723229a185536e250860"
server_key_version: "encryptionkey@0"
I20260808 22:57:37.865006 17855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:37.886476 17855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:37.889079 17855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:37.889261 17855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:37.893585 17855 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:45295
I20260808 22:57:37.893627 17925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:45295 every 8 connection(s)
I20260808 22:57:37.894066 17855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:37.895797 17926 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:57:37.896288 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17855
I20260808 22:57:37.896587 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:33717
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=42647
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:57:37.896618 17926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap starting.
I20260808 22:57:37.905927 17926 log.cc:824] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:37.908385 17926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:57:37.908737 17926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0: Bootstrap complete.
I20260808 22:57:37.911198 17926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:37.911473 17926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Initialized, Role: FOLLOWER
I20260808 22:57:37.911671 17926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:37.911755 17926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:37.911813 17926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:37.911862 17926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:57:37.913062 17926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:37.913195 17926 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd0102b3e3246b79e6d4db3d64186e0; no voters:
I20260808 22:57:37.913420 17926 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:57:37.913471 17930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:57:37.913640 17930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [term 3 LEADER]: Becoming Leader. State: Replica: 0dd0102b3e3246b79e6d4db3d64186e0, State: Running, Role: LEADER
I20260808 22:57:37.913734 17926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:37.913810 17930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } }
I20260808 22:57:37.914158 17931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:37.914388 17931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:37.914160 17930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd0102b3e3246b79e6d4db3d64186e0. Latest consensus state: current_term: 3 leader_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd0102b3e3246b79e6d4db3d64186e0" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 45295 } } }
I20260808 22:57:37.914974 17930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd0102b3e3246b79e6d4db3d64186e0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:37.915241 17942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:37.916114 17942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:37.916257 17942 catalog_manager.cc:1259] Loaded cluster ID: 4312cd12402b484abde8350f8f15faeb
I20260808 22:57:37.916297 17942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 22:57:37.916610 17942 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3222daa9ad google::LogMessage::Fail() at ??:0
@ 0x7f3222dad88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3222daa4de google::LogMessage::Flush() at ??:0
@ 0x7f3222daba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f322accb0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f322acd48b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3224c780d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3224c6d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3225ad16db start_thread at ??:0
@ 0x7f32218a861f clone at ??:0
W20260808 22:57:38.024107 17928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:38.024427 17928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:38.024472 17928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:38.026264 17928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:38.026350 17928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:38.026403 17928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:38.028810 17928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:33717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=42647
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:38.029114 17928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:38.029421 17928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:38.035207 17965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:38.035183 17962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:38.035184 17963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:38.035835 17928 server_base.cc:1034] running on GCE node
I20260808 22:57:38.036098 17928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:38.036391 17928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:38.037561 17928 hybrid_clock.cc:648] HybridClock initialized: now 1786229858037539 us; error 38 us; skew 500 ppm
Aug 08 22:57:38 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:38.040354 17928 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:38.040918 17928 webserver.cc:460] Webserver started at http://127.1.212.1:42647/ using document root <none> and password file <none>
I20260808 22:57:38.041188 17928 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:38.041258 17928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:38.042809 17928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:57:38.043648 17972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.043835 17928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:38.043948 17928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "5b507642608f49a58f791a5489604ce1"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "3ee6fe39a0ff5a41de5bf9297b214c3f"
server_key_iv: "f2077deb934ffe506c992030d437d833"
server_key_version: "encryptionkey@0"
I20260808 22:57:38.044371 17928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:38.055212 17928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:38.057794 17928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:38.057982 17928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:38.058259 17928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:38.058775 17979 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:57:38.061183 17928 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:57:38.061290 17928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 22:57:38.061343 17928 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:57:38.062080 17928 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:57:38.062142 17928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:38.062227 17979 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Bootstrap starting.
I20260808 22:57:38.064648 17979 log.cc:824] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:38.068585 17979 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:57:38.069051 17979 tablet_bootstrap.cc:492] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Bootstrap complete.
I20260808 22:57:38.069236 17979 ts_tablet_manager.cc:1397] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260808 22:57:38.069806 17928 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:33717
I20260808 22:57:38.069955 18094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:33717 every 8 connection(s)
I20260808 22:57:38.070232 17928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:38.071246 17979 raft_consensus.cc:348] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:38.071714 17979 raft_consensus.cc:729] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b507642608f49a58f791a5489604ce1, State: Initialized, Role: FOLLOWER
I20260808 22:57:38.071900 17979 consensus_queue.cc:260] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:38.072232 17979 ts_tablet_manager.cc:1428] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 22:57:38.073644 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 17928
I20260808 22:57:38.073913 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:38439
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=45837
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:38.189426 18099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:38.189779 18099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:38.189813 18099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:38.191241 18099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:38.191289 18099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:38.191322 18099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:38.193075 18099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:38439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=45837
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:38.193375 18099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:38.193710 18099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:38.194619 18097 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:57:38.071070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:37535)
0808 22:57:38.071243 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.071289 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:38.071570 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:38.194371 (+122801us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260808 22:57:38.194893 18095 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:45295 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:45295: Client connection negotiation failed: client connection to 127.1.212.62:45295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:57:38.195263 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.195151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:43935)
0808 22:57:38.195199 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.195240 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 22:57:38.195614 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.195519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:47479)
0808 22:57:38.195557 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.195591 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 22:57:38.195753 18095 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:57:38.199514 18108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:38.199520 18106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:38.199575 18099 server_base.cc:1034] running on GCE node
W20260808 22:57:38.199695 18105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:38.199926 18099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:38.200203 18099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:38.201375 18099 hybrid_clock.cc:648] HybridClock initialized: now 1786229858201361 us; error 30 us; skew 500 ppm
Aug 08 22:57:38 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:38.204057 18099 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:38.204552 18099 webserver.cc:460] Webserver started at http://127.1.212.2:45837/ using document root <none> and password file <none>
I20260808 22:57:38.204759 18099 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:38.204912 18099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:38.206323 18099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:38.206943 18115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.207129 18099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:38.207226 18099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "ce8c98fc1a114649b56dfc9febc17d8d"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "2324c26d4b86b37c80f9b255baa71a81"
server_key_iv: "ab2c12aa17458fd19b1967a3d8d8604d"
server_key_version: "encryptionkey@0"
I20260808 22:57:38.207604 18099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:38.216395 18099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:38.218647 18099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:38.218787 18099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:38.219003 18099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:38.219412 18099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:38.219452 18099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.219476 18099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:38.219488 18099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.225824 18099 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:38439
I20260808 22:57:38.225932 18234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:38439 every 8 connection(s)
I20260808 22:57:38.226274 18099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:57:38.227388 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.226983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:52285)
0808 22:57:38.227153 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.227229 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260808 22:57:38.227800 18236 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:45295 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:45295: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
I20260808 22:57:38.228093 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.227991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:51219)
0808 22:57:38.228044 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.228074 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:57:38.228364 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.228282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:43443)
0808 22:57:38.228323 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.228345 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 22:57:38.228469 18236 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:57:38.231762 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18099
I20260808 22:57:38.231989 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:34743
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:45295
--webserver_port=35841
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:38.319427 18239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:38.319671 18239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:38.319700 18239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:38.321077 18239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:38.321146 18239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:38.321189 18239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:38.322813 18239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:34743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=35841
--tserver_master_addrs=127.1.212.62:45295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:38.323048 18239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:38.323261 18239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:38.328895 18244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:38.328922 18239 server_base.cc:1034] running on GCE node
W20260808 22:57:38.328936 18245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:38.328895 18247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:38.329442 18239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:38.329727 18239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:38.330891 18239 hybrid_clock.cc:648] HybridClock initialized: now 1786229858330883 us; error 35 us; skew 500 ppm
Aug 08 22:57:38 dist-test-slave-1x4g krb5kdc[16728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:38.333992 18239 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:38.334470 18239 webserver.cc:460] Webserver started at http://127.1.212.3:35841/ using document root <none> and password file <none>
I20260808 22:57:38.334738 18239 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:38.334807 18239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:38.336241 18239 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:38.336923 18254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.337105 18239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:38.337208 18239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "c8fd095566cd4133b2f9338226981784"
format_stamp: "Formatted at 2026-08-08 22:57:36 on dist-test-slave-1x4g"
server_key: "5d763eb18e22d55f0af5a3a2ccb6cca2"
server_key_iv: "379abcde534d50a983198ea2e2c40b73"
server_key_version: "encryptionkey@0"
I20260808 22:57:38.337589 18239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:38.352541 18239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:38.355237 18239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:38.355429 18239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:38.355706 18239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:38.356046 18239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:38.356101 18239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.356145 18239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:38.356175 18239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:38.362707 18239 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:34743
I20260808 22:57:38.362761 18371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:34743 every 8 connection(s)
I20260808 22:57:38.363226 18239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:57:38.364651 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.364195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:60579)
0808 22:57:38.364387 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.364486 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
W20260808 22:57:38.365212 18372 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:45295 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:45295: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
I20260808 22:57:38.365571 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.365445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:33911)
0808 22:57:38.365501 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.365541 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 22:57:38.365952 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:38.365827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:39997)
0808 22:57:38.365893 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:38.365924 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:57:38.366106 18372 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:57:38.368330 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18239
I20260808 22:57:39.196449 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.196181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:55283)
0808 22:57:39.196336 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:39.196382 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260808 22:57:39.229188 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.228939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:43739)
0808 22:57:39.229073 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:39.229122 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260808 22:57:39.366838 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.366562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:54881)
0808 22:57:39.366707 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:39.366768 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260808 22:57:39.681452 18380 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:39.681592 18380 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:39.682052 18380 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
Aug 08 22:57:39 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.3@KRBTEST.COM
Aug 08 22:57:39 dist-test-slave-1x4g krb5kdc[16728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.2@KRBTEST.COM
I20260808 22:57:39.696316 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.682374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:55519 (local address 127.1.212.2:38439)
0808 22:57:39.682584 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:39.682594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:39.682655 (+ 61us) server_negotiation.cc:407] Connection header received
0808 22:57:39.683296 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:39.683299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:39.683359 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:39.683534 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:39.684706 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.686161 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.687833 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.688500 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.688628 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:39.692151 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:39.692155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:39.692156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:39.692170 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:39.694469 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:39.694910 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:39.694913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:39.694914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:39.695066 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:39.695351 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:39.695351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:39.695352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:39.695615 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:39.695721 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:39.696029 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 22:57:39.696186 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260808 22:57:39.696316 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.682360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:34743 (local address 127.1.212.1:47485)
0808 22:57:39.682673 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:39.682682 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:39.682730 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:39.683643 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:39.683647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:39.684131 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:39.684509 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:39.684520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.686326 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.686328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:39.687714 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:39.687719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.688629 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.688631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:39.688833 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:39.689555 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:39.689567 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:39.691680 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:39.694311 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:39.694322 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:39.694325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:39.694694 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:39.695018 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:39.695019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:39.695020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:39.695185 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:39.695702 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:39.695712 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:39.695843 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 22:57:39.695982 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 22:57:39.696200 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":101,"thread_start_us":95,"threads_started":1}
I20260808 22:57:39.696375 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.682547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:47485 (local address 127.1.212.3:34743)
0808 22:57:39.682719 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:39.682727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:39.682757 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:57:39.683295 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:39.683299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:39.683359 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:39.683533 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:39.684691 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.686162 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.687833 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.688506 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.688629 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:39.692162 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:39.692166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:39.692167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:39.692179 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:39.694203 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:39.694798 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:39.694802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:39.694802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:39.694938 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:39.695276 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:39.695277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:39.695278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:39.695616 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:39.695722 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:39.696060 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 22:57:39.696249 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260808 22:57:39.696434 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:39.682245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:38439 (local address 127.1.212.1:55519)
0808 22:57:39.682290 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:39.682304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:39.682368 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:39.683643 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:39.683647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:39.684529 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:39.684597 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:39.684599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.686340 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.686341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:39.687714 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:39.687719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:39.688631 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:39.688632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:39.688833 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:39.689554 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:39.689567 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:39.692025 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:39.694559 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:39.694560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:39.694560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:39.694814 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:39.695164 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:39.695165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:39.695166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:39.695269 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:39.695701 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:39.695709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:39.695842 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 22:57:39.695968 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:57:39.696284 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":144}
W20260808 22:57:39.697283 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
W20260808 22:57:39.697747 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:39.697815 17973 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b507642608f49a58f791a5489604ce1; no voters: c8fd095566cd4133b2f9338226981784, ce8c98fc1a114649b56dfc9febc17d8d
I20260808 22:57:39.697974 18380 raft_consensus.cc:2726] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:57:40.197283 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:40.197038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:40653)
0808 22:57:40.197169 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:40.197211 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260808 22:57:40.229897 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:40.229657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:52629)
0808 22:57:40.229808 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:40.229853 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260808 22:57:40.367745 18387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:40.367477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:57339)
0808 22:57:40.367647 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:40.367692 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260808 22:57:41.198143 18388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:41.197830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:58771)
0808 22:57:41.198032 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:41.198090 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":96,"threads_started":1}
I20260808 22:57:41.230643 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:41.230403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:49415)
0808 22:57:41.230550 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:41.230592 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260808 22:57:41.336768 18390 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:41.336961 18390 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:41.337162 18390 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
W20260808 22:57:41.337674 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
W20260808 22:57:41.337772 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:41.337860 17973 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b507642608f49a58f791a5489604ce1; no voters: c8fd095566cd4133b2f9338226981784, ce8c98fc1a114649b56dfc9febc17d8d
I20260808 22:57:41.337986 18390 raft_consensus.cc:2726] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:57:41.368489 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:41.368261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:41655)
0808 22:57:41.368405 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:41.368440 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260808 22:57:42.199105 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:42.198818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:46361)
0808 22:57:42.199002 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:42.199044 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260808 22:57:42.231505 18393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:42.231202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:36349)
0808 22:57:42.231404 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:42.231451 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260808 22:57:42.369511 18394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:42.369134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:43177)
0808 22:57:42.369385 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:42.369444 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":104,"threads_started":1}
I20260808 22:57:42.977082 18395 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:42.977186 18395 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:42.977320 18395 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
W20260808 22:57:42.977764 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
W20260808 22:57:42.977856 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:42.977888 17973 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b507642608f49a58f791a5489604ce1; no voters: c8fd095566cd4133b2f9338226981784, ce8c98fc1a114649b56dfc9febc17d8d
I20260808 22:57:42.977998 18395 raft_consensus.cc:2726] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:57:43.200127 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:43.199851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:39673)
0808 22:57:43.200018 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:43.200068 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260808 22:57:43.232332 18397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:43.232085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:50589)
0808 22:57:43.232227 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:43.232282 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260808 22:57:43.370476 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:43.370127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:51007)
0808 22:57:43.370306 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:43.370404 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260808 22:57:44.201118 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:44.200768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:35287)
0808 22:57:44.201019 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:44.201068 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
I20260808 22:57:44.233237 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:44.232975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:42987)
0808 22:57:44.233141 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:44.233187 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260808 22:57:44.371389 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:44.371087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:36535)
0808 22:57:44.371269 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:44.371336 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260808 22:57:45.202204 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:45.201883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:55185)
0808 22:57:45.202097 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:45.202151 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
I20260808 22:57:45.234191 18403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:45.233911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:42099)
0808 22:57:45.234097 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:45.234142 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
I20260808 22:57:45.372319 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:45.372015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:42579)
0808 22:57:45.372218 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:45.372266 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260808 22:57:46.203171 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:46.202906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:52399)
0808 22:57:46.203067 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:46.203117 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260808 22:57:46.235052 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:46.234781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:41361)
0808 22:57:46.234953 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:46.235003 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260808 22:57:46.373358 18407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:46.373088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:40525)
0808 22:57:46.373259 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:46.373307 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260808 22:57:46.389896 18408 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:46.390062 18408 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:46.390278 18408 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
W20260808 22:57:46.390709 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
W20260808 22:57:46.390820 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:46.390857 17973 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b507642608f49a58f791a5489604ce1; no voters: c8fd095566cd4133b2f9338226981784, ce8c98fc1a114649b56dfc9febc17d8d
I20260808 22:57:46.390949 18408 raft_consensus.cc:2726] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:57:47.204136 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:47.203880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:47619)
0808 22:57:47.204033 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:47.204085 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260808 22:57:47.235872 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:47.235617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:55581)
0808 22:57:47.235778 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:47.235821 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260808 22:57:47.374249 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:47.373926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:38297)
0808 22:57:47.374148 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:47.374200 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260808 22:57:48.205106 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:48.204844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:56297)
0808 22:57:48.205007 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:48.205054 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260808 22:57:48.236739 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:48.236437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:50545)
0808 22:57:48.236631 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:48.236686 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
I20260808 22:57:48.375145 18414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:48.374853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:58379)
0808 22:57:48.375033 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:48.375089 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260808 22:57:49.205927 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:49.205627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:55455)
0808 22:57:49.205805 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:49.205867 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260808 22:57:49.237579 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:49.237334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:53241)
0808 22:57:49.237486 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:49.237527 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260808 22:57:49.376058 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:49.375758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:55037)
0808 22:57:49.375957 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:49.376006 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
I20260808 22:57:50.206926 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:50.206643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:51839)
0808 22:57:50.206821 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:50.206869 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
I20260808 22:57:50.238384 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:50.238100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:42623)
0808 22:57:50.238285 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:50.238333 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260808 22:57:50.376989 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:50.376691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:43199)
0808 22:57:50.376889 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:50.376937 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":174,"threads_started":1}
I20260808 22:57:51.207769 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:51.207536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:32893)
0808 22:57:51.207673 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:51.207717 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260808 22:57:51.239261 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:51.239001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:55349)
0808 22:57:51.239169 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:51.239210 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260808 22:57:51.377787 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:51.377556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:57697)
0808 22:57:51.377694 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:51.377736 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260808 22:57:51.525286 18424 raft_consensus.cc:482] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:51.525518 18424 raft_consensus.cc:504] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce8c98fc1a114649b56dfc9febc17d8d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 38439 } } peers { permanent_uuid: "c8fd095566cd4133b2f9338226981784" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 34743 } } peers { permanent_uuid: "5b507642608f49a58f791a5489604ce1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 33717 } }
I20260808 22:57:51.525738 18424 leader_election.cc:288] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439), c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743)
W20260808 22:57:51.526217 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce8c98fc1a114649b56dfc9febc17d8d (127.1.212.2:38439): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
W20260808 22:57:51.526309 17973 leader_election.cc:341] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8fd095566cd4133b2f9338226981784 (127.1.212.3:34743): Not found: Tablet not found: be86e3bc6a414583bc66a352ca95fee0
I20260808 22:57:51.526347 17973 leader_election.cc:302] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b507642608f49a58f791a5489604ce1; no voters: c8fd095566cd4133b2f9338226981784, ce8c98fc1a114649b56dfc9febc17d8d
I20260808 22:57:51.526455 18424 raft_consensus.cc:2726] T be86e3bc6a414583bc66a352ca95fee0 P 5b507642608f49a58f791a5489604ce1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:57:52.208595 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:52.208374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:35557)
0808 22:57:52.208508 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:52.208543 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260808 22:57:52.240195 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:52.239868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:57677)
0808 22:57:52.240096 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:52.240136 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":116,"threads_started":1}
I20260808 22:57:52.378721 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:52.378421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:35695)
0808 22:57:52.378618 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:52.378669 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260808 22:57:53.209591 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:53.209300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.1:60993)
0808 22:57:53.209490 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:53.209541 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260808 22:57:53.241209 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:53.240939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.2:41197)
0808 22:57:53.241107 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:53.241158 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260808 22:57:53.372447 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 17928
I20260808 22:57:53.379561 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18099
I20260808 22:57:53.379611 18430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:53.379389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:45295 (local address 127.1.212.3:47287)
0808 22:57:53.379531 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:53.379564 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:45295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260808 22:57:53.387589 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18239
2026-08-08T22:57:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18686 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:57:53 dist-test-slave-1x4g krb5kdc[18433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:57:53 dist-test-slave-1x4g krb5kdc[18433](info): set up 2 sockets
Aug 08 22:57:53 dist-test-slave-1x4g krb5kdc[18433](info): commencing operation
krb5kdc: starting...
W20260808 22:57:54.475514 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:57:54 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:57:54.611948 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:54.701233 18446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:54.701485 18446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:54.701511 18446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:54.702891 18446 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:57:54.702950 18446 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:57:54.702965 18446 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:57:54.702976 18446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:57:54.702991 18446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:54.703009 18446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:54.703023 18446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:57:54.703037 18446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:57:54.704721 18446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:54.705047 18446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:54.705271 18446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:54.711176 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:54.711210 18446 server_base.cc:1034] running on GCE node
W20260808 22:57:54.711162 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:54.711169 18455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:54.711700 18446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:54.712070 18446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:54.713263 18446 hybrid_clock.cc:648] HybridClock initialized: now 1786229874713253 us; error 41 us; skew 500 ppm
Aug 08 22:57:54 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229874, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:54.716130 18446 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:57:54.716715 18446 webserver.cc:460] Webserver started at http://127.1.212.62:33365/ using document root <none> and password file <none>
I20260808 22:57:54.717007 18446 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:54.717093 18446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:54.717263 18446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:54.718514 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "3d39b621913f4d5ba69311b6da15cc60"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "4ff9e9bcf41aa08e253362b26dcdc9ce"
server_key_iv: "e0907ebfa4797c72592d0819b1ca3fd0"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.718941 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "3d39b621913f4d5ba69311b6da15cc60"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "4ff9e9bcf41aa08e253362b26dcdc9ce"
server_key_iv: "e0907ebfa4797c72592d0819b1ca3fd0"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.720494 18446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:57:54.721606 18463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:54.721974 18446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:54.722105 18446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "3d39b621913f4d5ba69311b6da15cc60"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "4ff9e9bcf41aa08e253362b26dcdc9ce"
server_key_iv: "e0907ebfa4797c72592d0819b1ca3fd0"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.722213 18446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:54.747812 18446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:54.750145 18446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:54.750332 18446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:54.754719 18446 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33879
I20260808 22:57:54.754812 18519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33879 every 8 connection(s)
I20260808 22:57:54.755177 18446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:57:54.756201 18520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:54.758810 18520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap starting.
I20260808 22:57:54.759019 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18446
I20260808 22:57:54.759128 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:57:54.759334 1872 external_mini_cluster.cc:1428] Setting key 65d3c396de308aa40f19489847e7e3e4
I20260808 22:57:54.759641 18520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:54.760054 18520 log.cc:824] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:54.760926 18520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: No bootstrap required, opened a new log
I20260808 22:57:54.763314 18520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:57:54.763514 18520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:54.763551 18520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Initialized, Role: FOLLOWER
I20260808 22:57:54.763687 18520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:57:54.763783 18520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:57:54.763816 18520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:57:54.763862 18520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:54.764668 18520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:57:54.764832 18520 leader_election.cc:302] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d39b621913f4d5ba69311b6da15cc60; no voters:
I20260808 22:57:54.765017 18520 leader_election.cc:288] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:57:54.765166 18525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:54.765329 18525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 LEADER]: Becoming Leader. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Running, Role: LEADER
I20260808 22:57:54.765342 18520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:57:54.766059 18525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
Aug 08 22:57:54 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:54.766875 18526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:57:54.766985 18526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
I20260808 22:57:54.767102 18528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d39b621913f4d5ba69311b6da15cc60. Latest consensus state: current_term: 1 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:57:54.767426 18528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
W20260808 22:57:54.767844 18540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:57:54.767912 18540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:57:54.767963 18542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:57:54.768182 18542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:57:54.771149 18542 catalog_manager.cc:1347] Generated new cluster ID: c4ab00cd80e3404db897cbdfb092dd8b
I20260808 22:57:54.771214 18542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:57:54.772550 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:54.759885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.1.212.62:33879)
0808 22:57:54.760073 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:54.760089 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:54.760117 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:57:54.760596 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:54.760603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:54.760651 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:54.760816 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:54.761646 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.762576 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.763718 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.764361 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.764486 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:54.767835 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:54.767840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:54.767841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:54.767855 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:54.770182 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:54.770673 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:54.770677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:54.770677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:54.770796 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:54.771053 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:54.771055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:54.771055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:54.771402 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:54.771571 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:54.772147 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 22:57:54.772303 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260808 22:57:54.777984 18542 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:57:54.778564 18542 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:57:54.780685 18542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Generated new TSK 0
I20260808 22:57:54.780933 18542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:57:54.816475 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:54.916666 18547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:54.916993 18547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:54.917024 18547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:54.918382 18547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:54.918435 18547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:54.918465 18547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:57:54.920100 18547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:54.920396 18547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:54.920679 18547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:54.926858 18552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:54.926863 18555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:54.926858 18553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:54.927275 18547 server_base.cc:1034] running on GCE node
I20260808 22:57:54.927572 18547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:54.927878 18547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:54.929138 18547 hybrid_clock.cc:648] HybridClock initialized: now 1786229874929121 us; error 34 us; skew 500 ppm
Aug 08 22:57:54 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229874, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:54.932003 18547 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:57:54.932585 18547 webserver.cc:460] Webserver started at http://127.1.212.1:43755/ using document root <none> and password file <none>
I20260808 22:57:54.932904 18547 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:54.932981 18547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:54.933159 18547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:54.934335 18547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "bb377a6b1b3542f1bb65542f4af81cee"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "868ce9234781f616e06f7ad09c0e458b"
server_key_iv: "3a473c5b0b5a2d0b1f8387f23ea2bb68"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.934751 18547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "bb377a6b1b3542f1bb65542f4af81cee"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "868ce9234781f616e06f7ad09c0e458b"
server_key_iv: "3a473c5b0b5a2d0b1f8387f23ea2bb68"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.936264 18547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:57:54.937253 18563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:54.937456 18547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:57:54.937579 18547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "bb377a6b1b3542f1bb65542f4af81cee"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "868ce9234781f616e06f7ad09c0e458b"
server_key_iv: "3a473c5b0b5a2d0b1f8387f23ea2bb68"
server_key_version: "encryptionkey@0"
I20260808 22:57:54.937667 18547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:54.947908 18547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:54.950295 18547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:54.950495 18547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:54.950789 18547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:54.951162 18547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:54.951225 18547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:54.951274 18547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:54.951303 18547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:54.958123 18547 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:39399
I20260808 22:57:54.958175 18684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:39399 every 8 connection(s)
I20260808 22:57:54.958552 18547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:57:54.964648 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18547
I20260808 22:57:54.964814 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:57:54.964993 1872 external_mini_cluster.cc:1428] Setting key aca6c3096dabdc3cca4550fab6246fa1
Aug 08 22:57:54 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229874, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:54.969699 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:54.959333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:49007 (local address 127.1.212.62:33879)
0808 22:57:54.959386 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:54.959389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:54.959691 (+ 302us) server_negotiation.cc:407] Connection header received
0808 22:57:54.960544 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:54.960566 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:54.960582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:54.960671 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:54.961748 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.962255 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.963317 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.963522 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.963583 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:54.966237 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:54.966239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:54.966239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:54.966249 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:54.968042 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:54.968527 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:54.968528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:54.968529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:54.968608 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:54.968957 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:54.968958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:54.968959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:54.969099 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:54.969181 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:54.969490 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:57:54.969603 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:57:54.969902 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:54.959319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:49007)
0808 22:57:54.959534 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:54.959589 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:54.960393 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:54.960761 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:54.960766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:54.961436 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:54.961651 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:54.961656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.962384 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.962386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:54.963225 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:54.963228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:54.963625 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:54.963626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:54.963747 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:54.964296 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:54.964306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:54.966135 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:54.968166 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:54.968176 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:54.968179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:54.968439 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:54.968691 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:54.968691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:54.968692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:54.968880 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:54.969172 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:54.969181 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:54.969343 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 22:57:54.969453 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:57:54.969635 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260808 22:57:54.970431 18685 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:57:54.970563 18685 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:54.970768 18685 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:57:54.971357 18481 ts_manager.cc:194] Registered new tserver with Master: bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399)
I20260808 22:57:54.972002 18481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:49007
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:57:54.996304 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:55.082792 18690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:55.083071 18690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:55.083115 18690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:55.084359 18690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:55.084450 18690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:55.084502 18690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:57:55.086123 18690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:55.086452 18690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:55.086711 18690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:57:55.092453 18698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:55.092442 18695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:55.092614 18690 server_base.cc:1034] running on GCE node
W20260808 22:57:55.092442 18696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:55.092903 18690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:55.093192 18690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:55.094403 18690 hybrid_clock.cc:648] HybridClock initialized: now 1786229875094379 us; error 31 us; skew 500 ppm
Aug 08 22:57:55 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229875, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:55.097277 18690 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:57:55.097795 18690 webserver.cc:460] Webserver started at http://127.1.212.2:44085/ using document root <none> and password file <none>
I20260808 22:57:55.098032 18690 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:55.098102 18690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:55.098258 18690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:55.099391 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "df42cfed9b914849942b0fd3bb89c8b6"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d4a3ff1ec76b8a1f7fecdfa2d4199557"
server_key_iv: "9e4a7f04334e2674f0f651fcc0bbf3df"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.099790 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "df42cfed9b914849942b0fd3bb89c8b6"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d4a3ff1ec76b8a1f7fecdfa2d4199557"
server_key_iv: "9e4a7f04334e2674f0f651fcc0bbf3df"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.101323 18690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:57:55.102243 18705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.102433 18690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:57:55.102555 18690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "df42cfed9b914849942b0fd3bb89c8b6"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d4a3ff1ec76b8a1f7fecdfa2d4199557"
server_key_iv: "9e4a7f04334e2674f0f651fcc0bbf3df"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.102636 18690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:55.116293 18690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:55.118917 18690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:55.119103 18690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:55.119380 18690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:55.119714 18690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:55.119766 18690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.119808 18690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:55.119841 18690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.126717 18690 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:32841
I20260808 22:57:55.126770 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:32841 every 8 connection(s)
I20260808 22:57:55.127135 18690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:57:55.132608 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18690
I20260808 22:57:55.132737 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:57:55.132966 1872 external_mini_cluster.cc:1428] Setting key fe89d534ed41a03555c6f588fe33bf7d
Aug 08 22:57:55 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229875, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:55.138340 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.127971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:59239 (local address 127.1.212.62:33879)
0808 22:57:55.128042 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.128044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.128226 (+ 182us) server_negotiation.cc:407] Connection header received
0808 22:57:55.128839 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.128840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.128856 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.128922 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:55.129983 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.130543 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.131634 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.131855 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.131951 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.134810 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:55.134812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:55.134813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:55.134824 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:55.136577 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.137220 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.137221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.137222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.137295 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.137551 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.137552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.137552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.137720 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:55.137794 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.138073 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.138196 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:57:55.138360 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.127927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:59239)
0808 22:57:55.128092 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.128134 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.128688 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.129006 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.129009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.129645 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:55.129859 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.129864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.130676 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.130677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.131532 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.131535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.131967 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.131968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.132107 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.132707 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:55.132722 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:55.134697 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:55.136693 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:55.136704 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:55.136707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:55.137113 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:55.137367 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:55.137368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:55.137368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:55.137484 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:55.137787 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:55.137793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:55.137931 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.138031 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.138175 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260808 22:57:55.139106 18824 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:57:55.139245 18824 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:55.139503 18824 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:57:55.139987 18481 ts_manager.cc:194] Registered new tserver with Master: df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
I20260808 22:57:55.140405 18481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:59239
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:57:55.164204 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:57:55.251308 18829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:57:55.251533 18829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:57:55.251559 18829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:57:55.252826 18829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:57:55.252871 18829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:57:55.252900 18829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:57:55.254467 18829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:57:55.254758 18829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:57:55.255018 18829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:57:55.261029 18829 server_base.cc:1034] running on GCE node
W20260808 22:57:55.261001 18837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:55.261019 18835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:57:55.261001 18834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:57:55.261579 18829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:57:55.261853 18829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:57:55.263029 18829 hybrid_clock.cc:648] HybridClock initialized: now 1786229875263021 us; error 35 us; skew 500 ppm
Aug 08 22:57:55 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229875, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:57:55.265897 18829 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:57:55.266492 18829 webserver.cc:460] Webserver started at http://127.1.212.3:44977/ using document root <none> and password file <none>
I20260808 22:57:55.266789 18829 fs_manager.cc:359] Metadata directory not provided
I20260808 22:57:55.266876 18829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:57:55.267040 18829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:57:55.268280 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "ae7a4b94d93c461bb39c7bb01ca141c2"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d7c86f0bf1df851bc7c1448cc52c40d9"
server_key_iv: "e6546dc503b8800475f5f8dfa64b7550"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.268689 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "ae7a4b94d93c461bb39c7bb01ca141c2"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d7c86f0bf1df851bc7c1448cc52c40d9"
server_key_iv: "e6546dc503b8800475f5f8dfa64b7550"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.270191 18829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:55.271040 18844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.271235 18829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:57:55.271346 18829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "ae7a4b94d93c461bb39c7bb01ca141c2"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d7c86f0bf1df851bc7c1448cc52c40d9"
server_key_iv: "e6546dc503b8800475f5f8dfa64b7550"
server_key_version: "encryptionkey@0"
I20260808 22:57:55.271432 18829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:57:55.279671 18829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:57:55.281992 18829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:57:55.282173 18829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:57:55.282418 18829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:57:55.282752 18829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:57:55.282809 18829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.282852 18829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:57:55.282876 18829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:57:55.289364 18829 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:46721
I20260808 22:57:55.289413 18961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:46721 every 8 connection(s)
I20260808 22:57:55.289786 18829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:57:55.290875 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 18829
I20260808 22:57:55.290956 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:57:55.291115 1872 external_mini_cluster.cc:1428] Setting key fde24521dbf5af31edeb6ea6ef066af3
Aug 08 22:57:55 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229875, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:57:55.299957 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.290546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:39307 (local address 127.1.212.62:33879)
0808 22:57:55.290597 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.290599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.290817 (+ 218us) server_negotiation.cc:407] Connection header received
0808 22:57:55.291572 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.291574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.291586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.291649 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:55.292830 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.293354 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.294402 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.294600 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.294654 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.297072 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:55.297073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:55.297074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:55.297080 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:55.298537 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.298992 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.298993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.298993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.299050 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.299342 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.299342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.299343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.299455 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:55.299506 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.299812 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.299884 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:57:55.300217 18965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.290501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:39307)
0808 22:57:55.290688 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.290735 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.291423 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.291730 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.291736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.292472 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:57:55.292696 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.292702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.293481 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.293482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.294315 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.294316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.294681 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.294682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.294795 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.295420 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:57:55.295429 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:57:55.296968 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:57:55.298638 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:55.298648 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:55.298651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:55.298918 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:55.299143 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:57:55.299144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:57:55.299144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:57:55.299278 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:57:55.299531 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:57:55.299538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:57:55.299688 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.299784 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.299951 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260808 22:57:55.300793 18962 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:57:55.300902 18962 heartbeater.cc:464] Registering TS with master...
I20260808 22:57:55.301069 18962 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:57:55.301478 18481 ts_manager.cc:194] Registered new tserver with Master: ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721)
I20260808 22:57:55.301834 18481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:39307
I20260808 22:57:55.302898 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:57:55.310649 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.304330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.1.212.62:33879)
0808 22:57:55.304411 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.304414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.304498 (+ 84us) server_negotiation.cc:407] Connection header received
0808 22:57:55.304574 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.304575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.304585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.304635 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:57:55.305266 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.305810 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.306579 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.306812 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.306887 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.308130 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:57:55.308132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:57:55.308132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:57:55.308140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:57:55.309393 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.309752 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.309753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.309754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.309816 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:57:55.310026 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:57:55.310026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:57:55.310027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:57:55.310191 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:57:55.310303 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.310435 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.310524 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:57:55.312520 18481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:57:55.314251 18481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:57:55.321352 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.317259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56378 (local address 127.1.212.1:39399)
0808 22:57:55.317492 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.317496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.317506 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:57:55.317544 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.317547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.317560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.317862 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:55.317945 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.318627 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.320030 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.320750 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.320854 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.320925 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.321208 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.321260 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
I20260808 22:57:55.321339 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.317009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.0.0.1:56378)
0808 22:57:55.317263 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.317275 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.317333 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.317683 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.317684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.317686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:55.317769 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.317771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.318731 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.318733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.319937 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.319940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.320964 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.320966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.321077 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.321097 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.321176 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.321241 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260808 22:57:55.321534 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.316900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.1.212.3:46721)
0808 22:57:55.317473 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.317478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.317489 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:57:55.317534 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.317537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.317563 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.317642 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:55.317874 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.319055 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.320252 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.321001 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.321068 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.321136 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.321388 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.321440 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":99,"threads_started":1}
I20260808 22:57:55.321610 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.316769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:46721 (local address 127.0.0.1:56036)
0808 22:57:55.317006 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.317056 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.317130 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.317726 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.317726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.317728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:55.317786 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.317787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.319167 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.319168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.320167 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.320169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.321171 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.321172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.321237 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.321240 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.321287 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.321322 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260808 22:57:55.322263 18612 tablet_service.cc:1467] Processing CreateTablet for tablet 2782c05edc3645d8a48812243619ad80 (DEFAULT_TABLE table=test-table [id=332542fccbda4393b4c1d0e9a432fa69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:55.322412 18892 tablet_service.cc:1467] Processing CreateTablet for tablet 2782c05edc3645d8a48812243619ad80 (DEFAULT_TABLE table=test-table [id=332542fccbda4393b4c1d0e9a432fa69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:55.322676 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.316824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:32841 (local address 127.0.0.1:44510)
0808 22:57:55.317140 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.317150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.317193 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.317679 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.317682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.317685 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:55.317798 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.317800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.320242 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.320242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.320990 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.320992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.322396 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.322398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.322466 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.322471 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.322533 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.322586 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":29,"thread_start_us":101,"threads_started":1}
I20260808 22:57:55.322700 18612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2782c05edc3645d8a48812243619ad80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:55.322723 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.316953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.1.212.2:32841)
0808 22:57:55.317306 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.317310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.317319 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:55.317464 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.317469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.317489 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.317568 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:55.317890 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.319037 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.321094 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.322219 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.322297 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.322356 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.322584 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.322633 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":82,"threads_started":1}
I20260808 22:57:55.322767 18892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2782c05edc3645d8a48812243619ad80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:55.323410 18756 tablet_service.cc:1467] Processing CreateTablet for tablet 2782c05edc3645d8a48812243619ad80 (DEFAULT_TABLE table=test-table [id=332542fccbda4393b4c1d0e9a432fa69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:57:55.323742 18756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2782c05edc3645d8a48812243619ad80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:57:55.325776 18980 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Bootstrap starting.
I20260808 22:57:55.326488 18980 tablet_bootstrap.cc:654] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:55.326998 18981 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap starting.
I20260808 22:57:55.327109 18980 log.cc:824] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:55.327450 18982 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Bootstrap starting.
I20260808 22:57:55.327749 18981 tablet_bootstrap.cc:654] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:55.327955 18980 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: No bootstrap required, opened a new log
I20260808 22:57:55.328037 18980 ts_tablet_manager.cc:1397] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:57:55.328136 18982 tablet_bootstrap.cc:654] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Neither blocks nor log segments found. Creating new log.
I20260808 22:57:55.328359 18981 log.cc:824] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:55.328729 18982 log.cc:824] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:57:55.329288 18981 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: No bootstrap required, opened a new log
I20260808 22:57:55.329458 18981 ts_tablet_manager.cc:1397] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:57:55.329727 18982 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: No bootstrap required, opened a new log
I20260808 22:57:55.329806 18982 ts_tablet_manager.cc:1397] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:57:55.330506 18980 raft_consensus.cc:348] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.330749 18980 raft_consensus.cc:374] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:55.330819 18980 raft_consensus.cc:729] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7a4b94d93c461bb39c7bb01ca141c2, State: Initialized, Role: FOLLOWER
I20260808 22:57:55.331014 18980 consensus_queue.cc:260] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.331161 18981 raft_consensus.cc:348] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.331370 18980 ts_tablet_manager.cc:1428] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:55.331415 18962 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
I20260808 22:57:55.331485 18981 raft_consensus.cc:374] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:55.331521 18981 raft_consensus.cc:729] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb377a6b1b3542f1bb65542f4af81cee, State: Initialized, Role: FOLLOWER
I20260808 22:57:55.331635 18981 consensus_queue.cc:260] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.331954 18982 raft_consensus.cc:348] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.332069 18981 ts_tablet_manager.cc:1428] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:55.332125 18982 raft_consensus.cc:374] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:57:55.332161 18982 raft_consensus.cc:729] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df42cfed9b914849942b0fd3bb89c8b6, State: Initialized, Role: FOLLOWER
I20260808 22:57:55.332269 18982 consensus_queue.cc:260] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.332329 18685 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
I20260808 22:57:55.332603 18982 ts_tablet_manager.cc:1428] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:57:55.332759 18824 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
W20260808 22:57:55.377735 18825 tablet.cc:2369] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:57:55.459363 18686 tablet.cc:2369] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:57:55.460668 18986 raft_consensus.cc:482] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:57:55.460840 18986 raft_consensus.cc:504] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.461237 18986 leader_election.cc:288] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
I20260808 22:57:55.465461 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.461537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:58357 (local address 127.1.212.1:39399)
0808 22:57:55.461588 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.461591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.461605 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:57:55.461643 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.461644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.461656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.461717 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:55.461927 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.462744 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.464384 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.465049 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.465137 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.465149 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.465331 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.465379 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:57:55.465461 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.461558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:51225 (local address 127.1.212.2:32841)
0808 22:57:55.461619 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.461621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.461766 (+ 145us) server_negotiation.cc:407] Connection header received
0808 22:57:55.461803 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.461804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.461814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.461859 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:57:55.462059 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.462755 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.464394 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.465048 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.465137 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.465148 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.465331 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.465389 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:57:55.465495 18965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.461442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.1.212.3:58357)
0808 22:57:55.461505 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.461519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.461577 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.461744 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.461745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.461747 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:55.461833 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.461835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.462866 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.462867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.464244 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.464249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.465148 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.465150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.465242 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.465244 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.465307 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.465368 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":48}
I20260808 22:57:55.465502 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.461454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:32841 (local address 127.1.212.3:51225)
0808 22:57:55.461660 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.461669 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:57:55.461745 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:57:55.461888 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:57:55.461889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:57:55.461891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:57:55.461980 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.461982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.462866 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.462868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.464244 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:57:55.464249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.465148 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.465150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:57:55.465238 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.465243 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:57:55.465307 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:57:55.465368 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":105,"thread_start_us":121,"threads_started":1}
I20260808 22:57:55.465921 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2782c05edc3645d8a48812243619ad80" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb377a6b1b3542f1bb65542f4af81cee" is_pre_election: true
I20260808 22:57:55.466073 18635 raft_consensus.cc:2454] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 0.
I20260808 22:57:55.466361 18845 leader_election.cc:302] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae7a4b94d93c461bb39c7bb01ca141c2, bb377a6b1b3542f1bb65542f4af81cee; no voters:
I20260808 22:57:55.466409 18777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2782c05edc3645d8a48812243619ad80" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df42cfed9b914849942b0fd3bb89c8b6" is_pre_election: true
I20260808 22:57:55.466554 18777 raft_consensus.cc:2454] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 0.
I20260808 22:57:55.466518 18986 raft_consensus.cc:2781] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:57:55.466599 18986 raft_consensus.cc:482] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:57:55.466638 18986 raft_consensus.cc:3037] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:55.467463 18986 raft_consensus.cc:504] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.467587 18986 leader_election.cc:288] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 election: Requested vote from peers bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
I20260808 22:57:55.467775 18777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2782c05edc3645d8a48812243619ad80" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df42cfed9b914849942b0fd3bb89c8b6"
I20260808 22:57:55.467854 18777 raft_consensus.cc:3037] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:55.468590 18777 raft_consensus.cc:2454] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 1.
I20260808 22:57:55.468731 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2782c05edc3645d8a48812243619ad80" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb377a6b1b3542f1bb65542f4af81cee"
I20260808 22:57:55.468865 18847 leader_election.cc:302] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6; no voters:
I20260808 22:57:55.468859 18635 raft_consensus.cc:3037] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:57:55.469012 18986 raft_consensus.cc:2781] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:57:55.469228 18986 raft_consensus.cc:686] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 LEADER]: Becoming Leader. State: Replica: ae7a4b94d93c461bb39c7bb01ca141c2, State: Running, Role: LEADER
I20260808 22:57:55.469347 18986 consensus_queue.cc:237] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:57:55.469869 18635 raft_consensus.cc:2454] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 1.
I20260808 22:57:55.470371 18481 catalog_manager.cc:5495] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3). New cstate: current_term: 1 leader_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } health_report { overall_health: UNKNOWN } } }
I20260808 22:57:55.503629 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.501222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.1.212.3:46721)
0808 22:57:55.501283 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:57:55.501289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:57:55.501298 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:57:55.501340 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:57:55.501341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:57:55.501364 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:57:55.501442 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:57:55.501583 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.502047 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.502853 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:57:55.503043 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:57:55.503106 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:57:55.503312 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:57:55.503422 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:57:55.503481 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:57:55.503518 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 22:57:55.503536 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:57:55.505792 18777 raft_consensus.cc:1261] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Refusing update from remote peer ae7a4b94d93c461bb39c7bb01ca141c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:55.505887 18635 raft_consensus.cc:1261] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Refusing update from remote peer ae7a4b94d93c461bb39c7bb01ca141c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:57:55.506294 18986 consensus_queue.cc:1035] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:55.506465 18990 consensus_queue.cc:1035] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:57:55.509186 18994 mvcc.cc:205] Tried to move back new op lower bound from 7316397570068721664 to 7316397569923461120. Current Snapshot: MvccSnapshot[applied={T|T < 7316397570068721664}]
I20260808 22:57:55.509987 18995 mvcc.cc:205] Tried to move back new op lower bound from 7316397570068721664 to 7316397569923461120. Current Snapshot: MvccSnapshot[applied={T|T < 7316397570068721664 or (T in {7316397570068721664})}]
I20260808 22:57:55.512408 18479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:55.512516 18479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:57:55.513404 18479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Sending DeleteTablet for 3 replicas of tablet 2782c05edc3645d8a48812243619ad80
I20260808 22:57:55.513731 18892 tablet_service.cc:1514] Processing DeleteTablet for tablet 2782c05edc3645d8a48812243619ad80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:56036
I20260808 22:57:55.513792 18756 tablet_service.cc:1514] Processing DeleteTablet for tablet 2782c05edc3645d8a48812243619ad80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:44510
I20260808 22:57:55.513819 18612 tablet_service.cc:1514] Processing DeleteTablet for tablet 2782c05edc3645d8a48812243619ad80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:56378
I20260808 22:57:55.514039 19004 tablet_replica.cc:330] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: stopping tablet replica
I20260808 22:57:55.514115 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18547
I20260808 22:57:55.514179 19005 tablet_replica.cc:330] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: stopping tablet replica
I20260808 22:57:55.514206 19004 raft_consensus.cc:2229] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:57:55.514288 19005 raft_consensus.cc:2229] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:57:55.514348 19004 raft_consensus.cc:2258] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:55.514396 19005 raft_consensus.cc:2258] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:57:55.517635 19005 ts_tablet_manager.cc:1905] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:55.519876 19005 ts_tablet_manager.cc:1918] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:55.519995 19005 log.cc:1196] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/2782c05edc3645d8a48812243619ad80
I20260808 22:57:55.520309 19005 ts_tablet_manager.cc:1939] T 2782c05edc3645d8a48812243619ad80 P df42cfed9b914849942b0fd3bb89c8b6: Deleting consensus metadata
I20260808 22:57:55.520566 19004 ts_tablet_manager.cc:1905] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:57:55.521209 18466 catalog_manager.cc:4841] TS df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): tablet 2782c05edc3645d8a48812243619ad80 (table test-table [id=332542fccbda4393b4c1d0e9a432fa69]) successfully deleted
I20260808 22:57:55.522562 19004 ts_tablet_manager.cc:1918] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:57:55.522743 19004 log.cc:1196] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/2782c05edc3645d8a48812243619ad80
I20260808 22:57:55.523119 19004 ts_tablet_manager.cc:1939] T 2782c05edc3645d8a48812243619ad80 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting consensus metadata
I20260808 22:57:55.523722 18464 catalog_manager.cc:4841] TS ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): tablet 2782c05edc3645d8a48812243619ad80 (table test-table [id=332542fccbda4393b4c1d0e9a432fa69]) successfully deleted
W20260808 22:57:55.527175 18464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:39399 (error 108)
I20260808 22:57:55.527361 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18690
I20260808 22:57:55.528899 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:57:55.528709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.0.0.1:56394)
0808 22:57:55.528802 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:57:55.528850 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:39399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:57:55.529162 18464 catalog_manager.cc:4584] TS bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2782c05edc3645d8a48812243619ad80: Network error: Client connection negotiation failed: client connection to 127.1.212.1:39399: connect: Connection refused (error 111)
I20260808 22:57:55.534760 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18829
I20260808 22:57:55.541734 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 18446
I20260808 22:58:15.549147 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--webserver_interface=127.1.212.62
--webserver_port=33365
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:15.639374 19009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:15.639616 19009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:15.639642 19009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:15.641012 19009 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:58:15.641062 19009 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:58:15.641077 19009 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:58:15.641089 19009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:15.641104 19009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:15.641117 19009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:15.641130 19009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:15.641144 19009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:15.642861 19009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=33365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:15.643196 19009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:15.643466 19009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:58:15.654130 19009 server_base.cc:1034] running on GCE node
W20260808 22:58:15.654129 19016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.654137 19015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.654137 19018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:15.654672 19009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:15.655066 19009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:15.656278 19009 hybrid_clock.cc:648] HybridClock initialized: now 1786229895656243 us; error 81 us; skew 500 ppm
Aug 08 22:58:15 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:15.665395 19009 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:58:15.666138 19009 webserver.cc:460] Webserver started at http://127.1.212.62:33365/ using document root <none> and password file <none>
I20260808 22:58:15.666419 19009 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:15.666505 19009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:15.668123 19009 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 22:58:15.669008 19025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:15.669225 19009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:15.669359 19009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "3d39b621913f4d5ba69311b6da15cc60"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "4ff9e9bcf41aa08e253362b26dcdc9ce"
server_key_iv: "e0907ebfa4797c72592d0819b1ca3fd0"
server_key_version: "encryptionkey@0"
I20260808 22:58:15.669847 19009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:15.687374 19009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:15.689989 19009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:15.690176 19009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:15.694947 19009 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33879
I20260808 22:58:15.694988 19079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33879 every 8 connection(s)
I20260808 22:58:15.695472 19009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:15.696205 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19009
I20260808 22:58:15.696501 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:39399
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=43755
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:58:15.697427 19080 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:58:15.698176 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap starting.
I20260808 22:58:15.708494 19080 log.cc:824] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:15.710969 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:58:15.711421 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap complete.
I20260808 22:58:15.714053 19080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:15.714370 19080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Initialized, Role: FOLLOWER
I20260808 22:58:15.714572 19080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:15.714663 19080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:15.714731 19080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:15.714788 19080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:58:15.715978 19080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:15.716132 19080 leader_election.cc:302] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d39b621913f4d5ba69311b6da15cc60; no voters:
I20260808 22:58:15.716364 19080 leader_election.cc:288] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:58:15.716411 19084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:58:15.716609 19084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 LEADER]: Becoming Leader. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Running, Role: LEADER
I20260808 22:58:15.716701 19080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:15.716818 19084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:15.717140 19084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d39b621913f4d5ba69311b6da15cc60. Latest consensus state: current_term: 2 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:58:15.717239 19084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:15.717149 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:58:15.717514 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:15.718883 19100 catalog_manager.cc:1259] Loaded cluster ID: c4ab00cd80e3404db897cbdfb092dd8b
I20260808 22:58:15.718952 19100 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: loading cluster ID for follower catalog manager: success
I20260808 22:58:15.720158 19100 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: acquiring CA information for follower catalog manager: success
I20260808 22:58:15.720631 19100 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:58:15.720721 19090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:15.721136 19090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:15.721251 19090 catalog_manager.cc:1259] Loaded cluster ID: c4ab00cd80e3404db897cbdfb092dd8b
I20260808 22:58:15.721287 19090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:15.721446 19090 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:15.721547 19090 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Loaded TSK: 0
I20260808 22:58:15.734180 19090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Generated new TSK 1
I20260808 22:58:15.734316 19090 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:58:15.790294 19082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:15.790592 19082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:15.790642 19082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:15.791988 19082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:15.792074 19082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:15.792122 19082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:15.793999 19082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:39399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=43755
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:15.794329 19082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:15.794608 19082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:15.800155 19107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.800292 19110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.800153 19108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:15.800607 19082 server_base.cc:1034] running on GCE node
I20260808 22:58:15.800875 19082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:15.801227 19082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:15.802438 19082 hybrid_clock.cc:648] HybridClock initialized: now 1786229895802417 us; error 32 us; skew 500 ppm
Aug 08 22:58:15 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:15.805850 19082 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:58:15.806481 19082 webserver.cc:460] Webserver started at http://127.1.212.1:43755/ using document root <none> and password file <none>
I20260808 22:58:15.806774 19082 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:15.806838 19082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:15.808378 19082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:58:15.809244 19118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:15.809592 19082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:15.809715 19082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "bb377a6b1b3542f1bb65542f4af81cee"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "868ce9234781f616e06f7ad09c0e458b"
server_key_iv: "3a473c5b0b5a2d0b1f8387f23ea2bb68"
server_key_version: "encryptionkey@0"
I20260808 22:58:15.810160 19082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:15.825690 19082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:15.828418 19082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:15.828620 19082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:15.828962 19082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:15.829537 19125 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:58:15.831108 19082 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:58:15.831176 19082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 22:58:15.831207 19082 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:58:15.831951 19082 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:58:15.832002 19082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:15.832053 19125 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap starting.
I20260808 22:58:15.840313 19082 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:39399
I20260808 22:58:15.840406 19237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:39399 every 8 connection(s)
I20260808 22:58:15.841095 19082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:15.843416 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19082
I20260808 22:58:15.843756 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:32841
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=44085
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:58:15.846530 19125 log.cc:824] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:15.849557 19125 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:58:15.849953 19125 tablet_bootstrap.cc:492] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap complete.
I20260808 22:58:15.851311 19125 ts_tablet_manager.cc:1397] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.011s
Aug 08 22:58:15 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:15.852625 19125 raft_consensus.cc:348] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:58:15.853325 19125 raft_consensus.cc:729] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb377a6b1b3542f1bb65542f4af81cee, State: Initialized, Role: FOLLOWER
I20260808 22:58:15.853538 19125 consensus_queue.cc:260] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } }
I20260808 22:58:15.853904 19125 ts_tablet_manager.cc:1428] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:58:15.857573 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.841915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:43467)
0808 22:58:15.842231 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:15.842278 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:15.842958 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:15.843645 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:15.843649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:15.844207 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:15.844495 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.844504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.846515 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.846517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.847965 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.847970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.849059 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.849061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.849221 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.849912 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:15.849923 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:15.852076 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:15.855018 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:15.855029 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:15.855032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:15.855421 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:15.855812 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:15.855813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:15.855814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:15.855975 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:15.856618 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:15.856626 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:15.856898 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 22:58:15.857032 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 22:58:15.857272 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":107,"threads_started":1}
I20260808 22:58:15.858191 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.842052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:43467 (local address 127.1.212.62:33879)
0808 22:58:15.842289 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:15.842300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:15.842383 (+ 83us) server_negotiation.cc:407] Connection header received
0808 22:58:15.843252 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:15.843260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:15.843326 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:15.843499 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:15.844652 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.846214 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.848114 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.848810 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.848928 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.852265 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:15.852271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:15.852272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:15.852289 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:15.854880 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:15.855558 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:15.855564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:15.855564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:15.855720 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:15.856060 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:15.856062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:15.856062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:15.856448 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:15.856550 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:15.857807 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0808 22:58:15.857958 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260808 22:58:15.858942 19238 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:58:15.859066 19238 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:15.859301 19238 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:58:15.860281 19043 ts_manager.cc:194] Registered new tserver with Master: bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399)
I20260808 22:58:15.861625 19043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:43467
I20260808 22:58:15.862707 19238 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
I20260808 22:58:15.869412 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.861962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.1.212.1:39399)
0808 22:58:15.865250 (+ 3288us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:15.865256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:15.865265 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:58:15.865303 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:15.865306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:15.865322 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:15.865414 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:15.865642 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.866622 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.868202 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.869004 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.869068 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.869113 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:15.869310 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:58:15.869346 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3249,"thread_start_us":108,"threads_started":1}
I20260808 22:58:15.869436 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.861020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.0.0.1:55576)
0808 22:58:15.864929 (+ 3909us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:15.865003 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:15.865100 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:15.865456 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:15.865460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:15.865463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:15.865553 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.865555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.866720 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.866721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.868087 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.868092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.869090 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.869091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.869188 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.869205 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:58:15.869287 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:58:15.869347 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3860,"thread_start_us":90,"threads_started":1}
I20260808 22:58:15.869927 19169 tablet_service.cc:1514] Processing DeleteTablet for tablet 2782c05edc3645d8a48812243619ad80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:57:55 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:55576
I20260808 22:58:15.870611 19247 tablet_replica.cc:330] stopping tablet replica
I20260808 22:58:15.870750 19247 raft_consensus.cc:2229] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:15.870838 19247 raft_consensus.cc:2258] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:15.871181 19247 ts_tablet_manager.cc:1905] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:15.873672 19247 ts_tablet_manager.cc:1918] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:15.873775 19247 log.cc:1196] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal/wals/2782c05edc3645d8a48812243619ad80
I20260808 22:58:15.874068 19247 ts_tablet_manager.cc:1939] T 2782c05edc3645d8a48812243619ad80 P bb377a6b1b3542f1bb65542f4af81cee: Deleting consensus metadata
I20260808 22:58:15.874689 19026 catalog_manager.cc:4841] TS bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399): tablet 2782c05edc3645d8a48812243619ad80 (table test-table [id=332542fccbda4393b4c1d0e9a432fa69]) successfully deleted
W20260808 22:58:15.944362 19242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:15.944602 19242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:15.944630 19242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:15.945942 19242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:15.946002 19242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:15.946038 19242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:15.947683 19242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:32841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=44085
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:15.947963 19242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:15.948223 19242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:15.954232 19252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.954288 19255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:15.954232 19253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:15.954496 19242 server_base.cc:1034] running on GCE node
I20260808 22:58:15.954823 19242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:15.955101 19242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:15.956282 19242 hybrid_clock.cc:648] HybridClock initialized: now 1786229895956259 us; error 38 us; skew 500 ppm
Aug 08 22:58:15 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:15.959098 19242 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:58:15.959605 19242 webserver.cc:460] Webserver started at http://127.1.212.2:44085/ using document root <none> and password file <none>
I20260808 22:58:15.959818 19242 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:15.959873 19242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:15.961344 19242 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:15.961931 19263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:15.962124 19242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:58:15.962250 19242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "df42cfed9b914849942b0fd3bb89c8b6"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d4a3ff1ec76b8a1f7fecdfa2d4199557"
server_key_iv: "9e4a7f04334e2674f0f651fcc0bbf3df"
server_key_version: "encryptionkey@0"
I20260808 22:58:15.962627 19242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:15.974610 19242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:15.977234 19242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:15.977388 19242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:15.977674 19242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:15.978030 19242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:15.978075 19242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:15.978119 19242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:15.978149 19242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:15.984614 19242 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:32841
I20260808 22:58:15.984668 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:32841 every 8 connection(s)
I20260808 22:58:15.985078 19242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:58:15 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:15.992769 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19242
I20260808 22:58:15.993172 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:46721
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=44977
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:58:15.995379 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.985843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:41629 (local address 127.1.212.62:33879)
0808 22:58:15.985922 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:15.985924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:15.986154 (+ 230us) server_negotiation.cc:407] Connection header received
0808 22:58:15.986733 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:15.986735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:15.986752 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:15.986804 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:15.987831 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.988347 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.989491 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.989722 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.989812 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.992119 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:15.992122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:15.992122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:15.992131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:15.993860 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:15.994312 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:15.994313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:15.994314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:15.994386 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:15.994652 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:15.994653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:15.994654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:15.994783 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:15.994866 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:15.995164 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 22:58:15.995268 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:58:15.995581 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:15.985836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:41629)
0808 22:58:15.986026 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:15.986066 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:15.986608 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:15.986877 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:15.986881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:15.987501 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:15.987715 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.987720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.988463 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.988464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.989387 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:15.989391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:15.989831 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:15.989832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:15.989966 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:15.990517 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:15.990526 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:15.991987 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:15.993973 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:15.993983 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:15.993985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:15.994226 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:15.994456 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:15.994457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:15.994457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:15.994573 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:15.994856 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:15.994864 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:15.995023 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 22:58:15.995142 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:58:15.995313 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260808 22:58:15.996441 19384 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:58:15.996549 19384 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:15.996853 19384 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:58:15.997334 19043 ts_manager.cc:194] Registered new tserver with Master: df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
I20260808 22:58:15.997751 19043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:41629
W20260808 22:58:16.081470 19387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:16.081743 19387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:16.081785 19387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:16.083099 19387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:16.083180 19387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:16.083226 19387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:16.084846 19387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:46721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=44977
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:16.085150 19387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:16.085417 19387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:16.090992 19392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:16.090901 19395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:16.090901 19393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:16.091020 19387 server_base.cc:1034] running on GCE node
I20260808 22:58:16.091444 19387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:16.091668 19387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:16.092859 19387 hybrid_clock.cc:648] HybridClock initialized: now 1786229896092827 us; error 50 us; skew 500 ppm
Aug 08 22:58:16 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:16.095474 19387 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:58:16.096007 19387 webserver.cc:460] Webserver started at http://127.1.212.3:44977/ using document root <none> and password file <none>
I20260808 22:58:16.096256 19387 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:16.096351 19387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:16.097913 19387 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:16.098623 19402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:16.098821 19387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:16.098937 19387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "ae7a4b94d93c461bb39c7bb01ca141c2"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d7c86f0bf1df851bc7c1448cc52c40d9"
server_key_iv: "e6546dc503b8800475f5f8dfa64b7550"
server_key_version: "encryptionkey@0"
I20260808 22:58:16.099352 19387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:16.117254 19387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:16.119683 19387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:16.119853 19387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:16.120165 19387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:16.120530 19387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:16.120568 19387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:16.120597 19387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:16.120610 19387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:16.126888 19387 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:46721
I20260808 22:58:16.126936 19517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:46721 every 8 connection(s)
I20260808 22:58:16.127353 19387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:58:16.130414 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19387
Aug 08 22:58:16 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:16.138063 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.130911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.1.212.62:33879)
0808 22:58:16.131045 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.131048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.131173 (+ 125us) server_negotiation.cc:407] Connection header received
0808 22:58:16.131262 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.131263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.131277 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.131363 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:16.132026 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.132520 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.133523 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.133737 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.133845 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.135177 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:16.135179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:16.135179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:16.135187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:16.136768 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.137184 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.137185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.137186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.137255 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.137467 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.137468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.137469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.137600 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:16.137802 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.137841 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.137935 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 22:58:16.138187 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.128251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:55547 (local address 127.1.212.62:33879)
0808 22:58:16.128302 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.128305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.128565 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:58:16.129107 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.129108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.129119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.129178 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:16.130186 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.130905 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.131998 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.132243 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.132327 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.134920 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:16.134922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:16.134922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:16.134930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:16.136665 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.137150 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.137152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.137152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.137216 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.137495 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.137496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.137497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.137613 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:16.137691 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.138001 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.138094 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":20}
I20260808 22:58:16.138445 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.128223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:55547)
0808 22:58:16.128415 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.128462 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:16.129003 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:16.129243 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:16.129247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:16.129867 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:16.130083 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.130088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.131026 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.131027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.131888 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.131890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.132341 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.132342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.132480 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.133101 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:16.133111 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:16.134791 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:16.136793 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:16.136803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:16.136806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:16.137061 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:16.137294 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:16.137295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:16.137295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:16.137421 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:16.137684 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:16.137692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:16.137853 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 22:58:16.137967 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 22:58:16.138165 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260808 22:58:16.138952 19518 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33879
I20260808 22:58:16.139078 19518 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:16.139281 19518 heartbeater.cc:511] Master 127.1.212.62:33879 requested a full tablet report, sending...
I20260808 22:58:16.139648 19042 ts_manager.cc:194] Registered new tserver with Master: ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721)
I20260808 22:58:16.140179 19042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:55547
I20260808 22:58:16.150264 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:58:16.153398 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.150736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.1.212.62:33879)
0808 22:58:16.150821 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.150822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.150875 (+ 53us) server_negotiation.cc:407] Connection header received
0808 22:58:16.150958 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.150958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.151001 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.151066 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:16.151226 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.151727 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.152540 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.152755 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.152925 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.153102 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:16.153159 (+ 57us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 22:58:16.153301 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45498: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:58:16.153497 19241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45498: token expired
W20260808 22:58:16.153651 18968 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.212.62:33879)
I20260808 22:58:16.160223 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.153941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45500 (local address 127.1.212.62:33879)
0808 22:58:16.154006 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.154007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.154038 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:58:16.154115 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.154116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.154125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.154176 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:16.154699 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.155169 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.155869 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.156046 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.156106 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.157565 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:16.157567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:16.157567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:16.157575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:16.159013 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.159365 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.159366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.159366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.159427 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:16.159653 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:16.159653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:16.159653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:16.159766 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:16.159845 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.160038 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.160111 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:58:16.161772 19042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:58:16.163473 19042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:58:16.166064 19169 tablet_service.cc:1467] Processing CreateTablet for tablet 5f65abf244b441a3be37143f7a131a39 (DEFAULT_TABLE table=test-table [id=443d72d17627489f865b3379793c801f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:16.166325 19169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f65abf244b441a3be37143f7a131a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:16.168111 19125 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap starting.
I20260808 22:58:16.168591 19125 tablet_bootstrap.cc:654] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:16.169292 19125 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: No bootstrap required, opened a new log
I20260808 22:58:16.169371 19125 ts_tablet_manager.cc:1397] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:58:16.169518 19125 raft_consensus.cc:348] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.169622 19125 raft_consensus.cc:374] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:16.169679 19125 raft_consensus.cc:729] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb377a6b1b3542f1bb65542f4af81cee, State: Initialized, Role: FOLLOWER
I20260808 22:58:16.169740 19125 consensus_queue.cc:260] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.169852 19125 ts_tablet_manager.cc:1428] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:58:16.169894 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.165580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:46721 (local address 127.0.0.1:59102)
0808 22:58:16.165701 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.165713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:16.165749 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:16.166168 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:16.166169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:16.166171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:16.166225 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.166226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.167534 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.167535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.168374 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.168378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.169601 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.169603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.169673 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.169679 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:58:16.169751 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:58:16.169810 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 22:58:16.170174 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.165715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.1.212.3:46721)
0808 22:58:16.165950 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.165956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.165969 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:58:16.166022 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.166024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.166044 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.166146 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:16.166300 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.167449 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.168567 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.169509 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.169589 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.169653 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.169782 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.170079 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260808 22:58:16.170329 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.165667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:32841 (local address 127.0.0.1:41264)
0808 22:58:16.166060 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.166073 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:16.166133 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:16.166492 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:16.166493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:16.166496 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:16.166543 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.166544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.167914 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.167915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.169040 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.169043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.170075 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.170077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.170145 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.170150 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:16.170202 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:58:16.170253 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":112,"threads_started":1}
I20260808 22:58:16.170562 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.165811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.1.212.2:32841)
0808 22:58:16.166018 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.166023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.166323 (+ 300us) server_negotiation.cc:407] Connection header received
0808 22:58:16.166373 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.166375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.166393 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.166475 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:16.166621 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.167790 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.169138 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.169899 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.169969 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.170036 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.170429 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.170482 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260808 22:58:16.170984 19451 tablet_service.cc:1467] Processing CreateTablet for tablet 5f65abf244b441a3be37143f7a131a39 (DEFAULT_TABLE table=test-table [id=443d72d17627489f865b3379793c801f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:16.171275 19314 tablet_service.cc:1467] Processing CreateTablet for tablet 5f65abf244b441a3be37143f7a131a39 (DEFAULT_TABLE table=test-table [id=443d72d17627489f865b3379793c801f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:16.171429 19451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f65abf244b441a3be37143f7a131a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:16.171627 19314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f65abf244b441a3be37143f7a131a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:16.176584 19530 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Bootstrap starting.
I20260808 22:58:16.177383 19530 tablet_bootstrap.cc:654] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:16.177973 19530 log.cc:824] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:16.178382 19529 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Bootstrap starting.
I20260808 22:58:16.178983 19530 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: No bootstrap required, opened a new log
I20260808 22:58:16.179075 19530 ts_tablet_manager.cc:1397] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:58:16.179203 19529 tablet_bootstrap.cc:654] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:16.179852 19529 log.cc:824] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:16.181005 19529 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: No bootstrap required, opened a new log
I20260808 22:58:16.181099 19529 ts_tablet_manager.cc:1397] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:58:16.181373 19530 raft_consensus.cc:348] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.181551 19530 raft_consensus.cc:374] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:16.181576 19530 raft_consensus.cc:729] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7a4b94d93c461bb39c7bb01ca141c2, State: Initialized, Role: FOLLOWER
I20260808 22:58:16.181731 19530 consensus_queue.cc:260] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.182087 19530 ts_tablet_manager.cc:1428] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:16.182129 19518 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
I20260808 22:58:16.183482 19529 raft_consensus.cc:348] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.183647 19529 raft_consensus.cc:374] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:16.183671 19529 raft_consensus.cc:729] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df42cfed9b914849942b0fd3bb89c8b6, State: Initialized, Role: FOLLOWER
I20260808 22:58:16.183759 19529 consensus_queue.cc:260] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.184000 19529 ts_tablet_manager.cc:1428] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:16.184150 19384 heartbeater.cc:503] Master 127.1.212.62:33879 was elected leader, sending a full tablet report...
W20260808 22:58:16.235685 19385 tablet.cc:2369] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:58:16.342103 19239 tablet.cc:2369] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:16.362838 19533 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:16.362993 19533 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.363592 19533 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841), bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399)
I20260808 22:58:16.368011 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.363982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:38695 (local address 127.1.212.2:32841)
0808 22:58:16.364039 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.364041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.364188 (+ 147us) server_negotiation.cc:407] Connection header received
0808 22:58:16.364219 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.364220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.364234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.364300 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:16.364497 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.365329 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.366974 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.367582 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.367673 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.367686 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.367883 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.367945 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:58:16.368011 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.363899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:57791 (local address 127.1.212.1:39399)
0808 22:58:16.363959 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.363962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.364021 (+ 59us) server_negotiation.cc:407] Connection header received
0808 22:58:16.364120 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.364121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.364133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.364226 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:16.364498 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.365316 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.366960 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.367598 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.367673 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.367685 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.367882 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.367928 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:58:16.368039 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.363894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:32841 (local address 127.1.212.3:38695)
0808 22:58:16.364098 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.364108 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:16.364165 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:16.364310 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:16.364311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:16.364312 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:16.364377 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.364378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.365430 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.365432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.366819 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.366824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.367689 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.367690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.367790 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.367795 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:16.367858 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:58:16.367924 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":48,"thread_start_us":131,"threads_started":1}
I20260808 22:58:16.368038 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.363856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.1.212.3:57791)
0808 22:58:16.363945 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.363959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:16.364041 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:16.364245 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:16.364245 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:16.364247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:16.364356 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.364358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.365430 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.365432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.366819 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:16.366824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.367689 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.367690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:16.367794 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.367796 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:58:16.367858 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:58:16.367925 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":66}
I20260808 22:58:16.368507 19335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f65abf244b441a3be37143f7a131a39" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df42cfed9b914849942b0fd3bb89c8b6" is_pre_election: true
I20260808 22:58:16.368610 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f65abf244b441a3be37143f7a131a39" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb377a6b1b3542f1bb65542f4af81cee" is_pre_election: true
I20260808 22:58:16.368721 19335 raft_consensus.cc:2454] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 0.
I20260808 22:58:16.368852 19190 raft_consensus.cc:2454] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 0.
I20260808 22:58:16.369035 19405 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6; no voters:
I20260808 22:58:16.369175 19533 raft_consensus.cc:2781] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:58:16.369241 19533 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:58:16.369266 19533 raft_consensus.cc:3037] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:16.370071 19533 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.370247 19533 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 election: Requested vote from peers df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841), bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399)
I20260808 22:58:16.370455 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f65abf244b441a3be37143f7a131a39" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb377a6b1b3542f1bb65542f4af81cee"
I20260808 22:58:16.370455 19335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f65abf244b441a3be37143f7a131a39" candidate_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df42cfed9b914849942b0fd3bb89c8b6"
I20260808 22:58:16.370546 19190 raft_consensus.cc:3037] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:16.370546 19335 raft_consensus.cc:3037] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:16.371454 19190 raft_consensus.cc:2454] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 1.
I20260808 22:58:16.371482 19335 raft_consensus.cc:2454] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7a4b94d93c461bb39c7bb01ca141c2 in term 1.
I20260808 22:58:16.371703 19403 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae7a4b94d93c461bb39c7bb01ca141c2, bb377a6b1b3542f1bb65542f4af81cee; no voters:
I20260808 22:58:16.371834 19533 raft_consensus.cc:2781] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:16.372006 19533 raft_consensus.cc:686] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 LEADER]: Becoming Leader. State: Replica: ae7a4b94d93c461bb39c7bb01ca141c2, State: Running, Role: LEADER
I20260808 22:58:16.372144 19533 consensus_queue.cc:237] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:16.373070 19043 catalog_manager.cc:5495] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3). New cstate: current_term: 1 leader_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } health_report { overall_health: UNKNOWN } } }
W20260808 22:58:16.378059 19519 tablet.cc:2369] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:16.400718 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.398417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.1.212.3:46721)
0808 22:58:16.398491 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:16.398493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:16.398502 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:58:16.398546 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:16.398547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:16.398571 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:16.398635 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:16.398756 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.399237 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.399964 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:16.400157 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:16.400214 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:16.400385 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:16.400510 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:58:16.400587 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:16.400606 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 22:58:16.400628 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:58:16.402791 19335 raft_consensus.cc:1261] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Refusing update from remote peer ae7a4b94d93c461bb39c7bb01ca141c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:16.402770 19190 raft_consensus.cc:1261] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Refusing update from remote peer ae7a4b94d93c461bb39c7bb01ca141c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:16.403236 19536 consensus_queue.cc:1035] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:16.403373 19533 consensus_queue.cc:1035] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:16.406109 19538 mvcc.cc:205] Tried to move back new op lower bound from 7316397655662727168 to 7316397655540998144. Current Snapshot: MvccSnapshot[applied={T|T < 7316397655662727168}]
I20260808 22:58:16.406872 19540 mvcc.cc:205] Tried to move back new op lower bound from 7316397655662727168 to 7316397655540998144. Current Snapshot: MvccSnapshot[applied={T|T < 7316397655662727168}]
I20260808 22:58:16.409142 19043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:16.409260 19043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:16.410265 19043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Sending DeleteTablet for 3 replicas of tablet 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:16.410619 19451 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f65abf244b441a3be37143f7a131a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:16 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:59102
I20260808 22:58:16.410670 19169 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f65abf244b441a3be37143f7a131a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:16 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:55576
I20260808 22:58:16.410679 19314 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f65abf244b441a3be37143f7a131a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:16 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:41264
I20260808 22:58:16.411001 19548 tablet_replica.cc:330] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: stopping tablet replica
I20260808 22:58:16.411039 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19082
I20260808 22:58:16.411115 19548 raft_consensus.cc:2229] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:58:16.411248 19548 raft_consensus.cc:2258] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:16.411775 19548 ts_tablet_manager.cc:1905] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:16.414237 19548 ts_tablet_manager.cc:1918] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:16.414323 19548 log.cc:1196] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/5f65abf244b441a3be37143f7a131a39
I20260808 22:58:16.414628 19548 ts_tablet_manager.cc:1939] T 5f65abf244b441a3be37143f7a131a39 P ae7a4b94d93c461bb39c7bb01ca141c2: Deleting consensus metadata
I20260808 22:58:16.414642 19550 tablet_replica.cc:330] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: stopping tablet replica
I20260808 22:58:16.414888 19550 raft_consensus.cc:2229] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:16.415103 19550 raft_consensus.cc:2258] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:16.415293 19026 catalog_manager.cc:4841] TS ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): tablet 5f65abf244b441a3be37143f7a131a39 (table test-table [id=443d72d17627489f865b3379793c801f]) successfully deleted
I20260808 22:58:16.415743 19550 ts_tablet_manager.cc:1905] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:16.417830 19550 ts_tablet_manager.cc:1918] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:16.417930 19550 log.cc:1196] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/5f65abf244b441a3be37143f7a131a39
I20260808 22:58:16.418258 19550 ts_tablet_manager.cc:1939] T 5f65abf244b441a3be37143f7a131a39 P df42cfed9b914849942b0fd3bb89c8b6: Deleting consensus metadata
I20260808 22:58:16.418785 19028 catalog_manager.cc:4841] TS df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): tablet 5f65abf244b441a3be37143f7a131a39 (table test-table [id=443d72d17627489f865b3379793c801f]) successfully deleted
W20260808 22:58:16.420202 19026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:39399 (error 108)
I20260808 22:58:16.420483 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19242
I20260808 22:58:16.421510 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:16.420640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:39399 (local address 127.0.0.1:55578)
0808 22:58:16.421405 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:16.421462 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:39399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":736}
W20260808 22:58:16.421892 19026 catalog_manager.cc:4584] TS bb377a6b1b3542f1bb65542f4af81cee (127.1.212.1:39399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f65abf244b441a3be37143f7a131a39: Network error: Client connection negotiation failed: client connection to 127.1.212.1:39399: connect: Connection refused (error 111)
I20260808 22:58:16.428725 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19387
I20260808 22:58:16.436152 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19009
I20260808 22:58:36.443917 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--webserver_interface=127.1.212.62
--webserver_port=33365
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:36.534215 19552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:36.534466 19552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:36.534492 19552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:36.535822 19552 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:58:36.535876 19552 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:58:36.535888 19552 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:58:36.535902 19552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:36.535914 19552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:36.535928 19552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:36.535940 19552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:36.535986 19552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:36.537837 19552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=33365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:36.538165 19552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:36.538437 19552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:58:36.544093 19552 server_base.cc:1034] running on GCE node
W20260808 22:58:36.544086 19557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.544090 19558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.544100 19560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:36.544561 19552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:36.544963 19552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:36.546140 19552 hybrid_clock.cc:648] HybridClock initialized: now 1786229916546135 us; error 39 us; skew 500 ppm
Aug 08 22:58:36 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:36.553944 19552 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:58:36.554522 19552 webserver.cc:460] Webserver started at http://127.1.212.62:33365/ using document root <none> and password file <none>
I20260808 22:58:36.554801 19552 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:36.554855 19552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:36.556480 19552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:58:36.557466 19567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:36.557718 19552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:36.557834 19552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "3d39b621913f4d5ba69311b6da15cc60"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "4ff9e9bcf41aa08e253362b26dcdc9ce"
server_key_iv: "e0907ebfa4797c72592d0819b1ca3fd0"
server_key_version: "encryptionkey@0"
I20260808 22:58:36.558276 19552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:36.574615 19552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:36.577072 19552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:36.577241 19552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:36.581854 19552 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33879
I20260808 22:58:36.581897 19620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33879 every 8 connection(s)
I20260808 22:58:36.582340 19552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:36.584005 19622 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:58:36.584743 19622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap starting.
I20260808 22:58:36.591482 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19552
I20260808 22:58:36.591837 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:39399
--local_ip_for_outbound_sockets=127.1.212.1
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=43755
--webserver_interface=127.1.212.1
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:58:36.594316 19622 log.cc:824] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:36.596992 19622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:58:36.597337 19622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Bootstrap complete.
I20260808 22:58:36.599773 19622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:36.600062 19622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Initialized, Role: FOLLOWER
I20260808 22:58:36.600288 19622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:36.600395 19622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:36.600453 19622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:36.600512 19622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:58:36.601712 19622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:36.601852 19622 leader_election.cc:302] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d39b621913f4d5ba69311b6da15cc60; no voters:
I20260808 22:58:36.602087 19622 leader_election.cc:288] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:58:36.602141 19626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:58:36.602346 19626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [term 3 LEADER]: Becoming Leader. State: Replica: 3d39b621913f4d5ba69311b6da15cc60, State: Running, Role: LEADER
I20260808 22:58:36.602399 19622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:36.602525 19626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } }
I20260808 22:58:36.602939 19627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:58:36.602993 19628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d39b621913f4d5ba69311b6da15cc60. Latest consensus state: current_term: 3 leader_uuid: "3d39b621913f4d5ba69311b6da15cc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d39b621913f4d5ba69311b6da15cc60" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33879 } } }
I20260808 22:58:36.603060 19627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:36.603088 19628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:36.603302 19634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:36.604257 19634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:36.604403 19634 catalog_manager.cc:1259] Loaded cluster ID: c4ab00cd80e3404db897cbdfb092dd8b
I20260808 22:58:36.604647 19634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:36.605950 19634 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:36.606153 19634 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Loaded TSK: 0
I20260808 22:58:36.606205 19634 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d39b621913f4d5ba69311b6da15cc60: Loaded TSK: 1
F20260808 22:58:36.606437 19634 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbf164aa9ad google::LogMessage::Fail() at ??:0
@ 0x7fbf164ad88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbf164aa4de google::LogMessage::Flush() at ??:0
@ 0x7fbf164aba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbf18c17104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbf18c14d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbf1e4041cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbf1e40448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbf1e404594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbf1e3cac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbf1e3d4ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbf183780d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbf1836d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbf191d16db start_thread at ??:0
@ 0x7fbf14fa861f clone at ??:0
W20260808 22:58:36.701606 19624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:36.701841 19624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:36.701869 19624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:36.703250 19624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:36.703296 19624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:36.703326 19624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:36.705082 19624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:39399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=43755
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:36.705368 19624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:36.705632 19624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:36.712018 19667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.712168 19669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.712011 19666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:36.712055 19624 server_base.cc:1034] running on GCE node
I20260808 22:58:36.712514 19624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:36.712852 19624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:36.714046 19624 hybrid_clock.cc:648] HybridClock initialized: now 1786229916714030 us; error 35 us; skew 500 ppm
Aug 08 22:58:36 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:36.717001 19624 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:58:36.717569 19624 webserver.cc:460] Webserver started at http://127.1.212.1:43755/ using document root <none> and password file <none>
I20260808 22:58:36.717824 19624 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:36.717900 19624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:36.719533 19624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:36.720382 19676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:36.720597 19624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:36.720716 19624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "bb377a6b1b3542f1bb65542f4af81cee"
format_stamp: "Formatted at 2026-08-08 22:57:54 on dist-test-slave-1x4g"
server_key: "868ce9234781f616e06f7ad09c0e458b"
server_key_iv: "3a473c5b0b5a2d0b1f8387f23ea2bb68"
server_key_version: "encryptionkey@0"
I20260808 22:58:36.721158 19624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:36.739612 19624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:36.742283 19624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:36.742450 19624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:36.742743 19624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:36.743263 19686 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:58:36.745436 19624 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:58:36.745510 19624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 22:58:36.745564 19624 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:58:36.746269 19624 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:58:36.746328 19624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:36.746409 19686 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap starting.
I20260808 22:58:36.754400 19624 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:39399
I20260808 22:58:36.754469 19799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:39399 every 8 connection(s)
I20260808 22:58:36.754920 19624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:36.758873 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19624
I20260808 22:58:36.759066 19686 log.cc:824] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:36.759307 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:32841
--local_ip_for_outbound_sockets=127.1.212.2
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=44085
--webserver_interface=127.1.212.2
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:58:36.762285 19686 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:58:36.762672 19686 tablet_bootstrap.cc:492] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Bootstrap complete.
I20260808 22:58:36.764104 19686 ts_tablet_manager.cc:1397] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.011s
I20260808 22:58:36.765764 19686 raft_consensus.cc:348] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:36.766129 19686 raft_consensus.cc:729] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb377a6b1b3542f1bb65542f4af81cee, State: Initialized, Role: FOLLOWER
I20260808 22:58:36.766319 19686 consensus_queue.cc:260] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:36.766635 19686 ts_tablet_manager.cc:1428] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260808 22:58:36.825985 19803 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:58:36.755696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:46899)
0808 22:58:36.755903 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.755954 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:36.756488 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:36.825776 (+ 69288us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
W20260808 22:58:36.826238 19800 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:33879 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:33879: Client connection negotiation failed: client connection to 127.1.212.62:33879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:58:36.826606 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:36.826473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:43905)
0808 22:58:36.826533 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.826575 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:58:36.831622 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:36.831474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:37445)
0808 22:58:36.831551 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.831582 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 22:58:36.831780 19800 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:58:36.854818 19805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:36.855096 19805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:36.855140 19805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:36.856389 19805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:36.856467 19805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:36.856515 19805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:36.858144 19805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:32841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=44085
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:36.858444 19805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:36.858686 19805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:36.864204 19813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.864233 19816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.864204 19814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:36.864522 19805 server_base.cc:1034] running on GCE node
I20260808 22:58:36.864817 19805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:36.865103 19805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:36.866266 19805 hybrid_clock.cc:648] HybridClock initialized: now 1786229916866260 us; error 38 us; skew 500 ppm
Aug 08 22:58:36 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:36.868867 19805 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:58:36.869313 19805 webserver.cc:460] Webserver started at http://127.1.212.2:44085/ using document root <none> and password file <none>
I20260808 22:58:36.869521 19805 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:36.869565 19805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:36.870926 19805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:36.871549 19823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:36.871711 19805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:58:36.871807 19805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "df42cfed9b914849942b0fd3bb89c8b6"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d4a3ff1ec76b8a1f7fecdfa2d4199557"
server_key_iv: "9e4a7f04334e2674f0f651fcc0bbf3df"
server_key_version: "encryptionkey@0"
I20260808 22:58:36.872182 19805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:36.883220 19805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:36.885787 19805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:36.885917 19805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:36.886133 19805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:36.886471 19805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:36.886516 19805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:36.886541 19805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:36.886554 19805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:36.892868 19805 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:32841
I20260808 22:58:36.892935 19942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:32841 every 8 connection(s)
I20260808 22:58:36.893337 19805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:58:36.894469 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:36.894077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:41461)
0808 22:58:36.894245 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.894320 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260808 22:58:36.894906 19943 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:33879 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:33879: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
I20260808 22:58:36.895201 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:36.895115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:57935)
0808 22:58:36.895155 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.895183 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 22:58:36.895570 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:36.895484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:35507)
0808 22:58:36.895530 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:36.895554 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:58:36.895757 19943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:58:36.896047 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19805
I20260808 22:58:36.896342 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:46721
--local_ip_for_outbound_sockets=127.1.212.3
--tserver_master_addrs=127.1.212.62:33879
--webserver_port=44977
--webserver_interface=127.1.212.3
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:36.984135 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:36.984387 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:36.984416 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:36.985656 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:36.985708 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:36.985737 19946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:36.987262 19946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:46721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=44977
--tserver_master_addrs=127.1.212.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:36.987486 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:36.987689 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:36.993085 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:36.993108 19952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:36.993268 19946 server_base.cc:1034] running on GCE node
W20260808 22:58:36.993110 19951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:36.993706 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:36.994009 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:36.995182 19946 hybrid_clock.cc:648] HybridClock initialized: now 1786229916995171 us; error 42 us; skew 500 ppm
Aug 08 22:58:36 dist-test-slave-1x4g krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:36.998185 19946 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:58:36.998704 19946 webserver.cc:460] Webserver started at http://127.1.212.3:44977/ using document root <none> and password file <none>
I20260808 22:58:36.998952 19946 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:36.999002 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:37.000602 19946 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:58:37.001550 19961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:37.001875 19946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:58:37.002027 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "ae7a4b94d93c461bb39c7bb01ca141c2"
format_stamp: "Formatted at 2026-08-08 22:57:55 on dist-test-slave-1x4g"
server_key: "d7c86f0bf1df851bc7c1448cc52c40d9"
server_key_iv: "e6546dc503b8800475f5f8dfa64b7550"
server_key_version: "encryptionkey@0"
I20260808 22:58:37.002453 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:37.034489 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:37.036871 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:37.037063 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:37.037361 19946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:37.037729 19946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:37.037791 19946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:37.037827 19946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:37.037871 19946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:37.044531 19946 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:46721
I20260808 22:58:37.044593 20082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:46721 every 8 connection(s)
I20260808 22:58:37.045070 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:58:37.046265 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:37.045824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:41969)
0808 22:58:37.046016 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:37.046102 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260808 22:58:37.046729 20083 heartbeater.cc:644] Failed to heartbeat to 127.1.212.62:33879 (0 consecutive failures): Network error: Failed to ping master at 127.1.212.62:33879: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
I20260808 22:58:37.047050 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:37.046940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:42375)
0808 22:58:37.047002 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:37.047032 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 22:58:37.047309 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:37.047257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:42959)
0808 22:58:37.047279 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:37.047298 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260808 22:58:37.047425 20083 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:58:37.053329 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 19946
I20260808 22:58:37.832592 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:37.832217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:36943)
0808 22:58:37.832431 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:37.832488 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":76,"threads_started":1}
I20260808 22:58:37.896409 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:37.896158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:34657)
0808 22:58:37.896300 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:37.896339 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260808 22:58:38.048094 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.047834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:33703)
0808 22:58:38.047987 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:38.048033 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260808 22:58:38.301087 20090 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:38.301236 20090 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:38.301719 20090 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
Aug 08 22:58:38 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.3@KRBTEST.COM
Aug 08 22:58:38 dist-test-slave-1x4g krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.2@KRBTEST.COM
I20260808 22:58:38.314183 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.302058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:33005 (local address 127.1.212.3:46721)
0808 22:58:38.302226 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:38.302245 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:38.302287 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:58:38.302765 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:38.302767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:38.302811 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:38.302938 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:38.303898 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.304945 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306143 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.306657 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306787 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:38.309536 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:38.309542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:38.309542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:38.309558 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:38.311750 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:38.312723 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:38.312726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:38.312726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:38.312901 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:38.313115 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:38.313116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:38.313117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:38.313403 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:38.313489 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:38.313899 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 22:58:38.314059 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":56,"threads_started":1}
I20260808 22:58:38.314186 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.301947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:46721 (local address 127.1.212.1:33005)
0808 22:58:38.302149 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:38.302161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:38.302223 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:38.303088 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:38.303089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:38.303526 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:38.303746 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:38.303754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.305059 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.305060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:38.306019 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:38.306023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.306770 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:38.306858 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:38.307478 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:38.307489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:38.309272 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:38.311870 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:38.311881 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:38.311884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:38.312395 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:38.312993 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:38.312993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:38.312997 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:38.313049 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:38.313477 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:38.313486 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:38.313684 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 22:58:38.313810 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:58:38.314056 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":48,"thread_start_us":120,"threads_started":1}
I20260808 22:58:38.314234 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.302063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:41825 (local address 127.1.212.2:32841)
0808 22:58:38.302252 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:38.302265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:38.302308 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:58:38.302769 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:38.302772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:38.302811 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:38.302982 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:38.303887 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.304951 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306127 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.306644 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306767 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:38.309796 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:38.309801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:38.309801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:38.309813 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:38.311890 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:38.312498 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:38.312502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:38.312502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:38.312621 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:38.312964 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:38.312965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:38.312966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:38.313376 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:38.313486 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:38.313899 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 22:58:38.314123 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260808 22:58:38.314237 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.301932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:32841 (local address 127.1.212.1:41825)
0808 22:58:38.302011 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:38.302024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:38.302084 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:38.303042 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:38.303045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:38.303527 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:38.303746 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:38.303754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.305084 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.305085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:38.306021 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:38.306024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:38.306735 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:38.306737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:38.306942 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:38.307642 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:38.307647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:38.309660 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:38.311971 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:38.311972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:38.311972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:38.312383 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:38.312697 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:38.312697 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:38.312698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:38.312868 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:38.313482 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:38.313486 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:38.313684 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 22:58:38.313810 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:58:38.314116 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":340}
W20260808 22:58:38.315179 19681 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
W20260808 22:58:38.315826 19678 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:38.315886 19678 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb377a6b1b3542f1bb65542f4af81cee; no voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6
I20260808 22:58:38.316004 20090 raft_consensus.cc:2726] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:58:38.833487 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.833217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:44099)
0808 22:58:38.833363 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:38.833413 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260808 22:58:38.897289 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:38.897037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:47693)
0808 22:58:38.897195 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:38.897241 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260808 22:58:39.049144 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:39.048899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:49079)
0808 22:58:39.049046 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:39.049092 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260808 22:58:39.834313 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:39.834047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:56333)
0808 22:58:39.834226 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:39.834264 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260808 22:58:39.897974 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:39.897762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:54471)
0808 22:58:39.897888 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:39.897926 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260808 22:58:40.049961 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:40.049680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:43889)
0808 22:58:40.049862 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:40.049908 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260808 22:58:40.192736 20101 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:40.192881 20101 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:40.193195 20101 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
W20260808 22:58:40.193668 19681 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
W20260808 22:58:40.193814 19678 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:40.193886 19678 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb377a6b1b3542f1bb65542f4af81cee; no voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6
I20260808 22:58:40.193981 20101 raft_consensus.cc:2726] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:58:40.835201 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:40.834917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:57847)
0808 22:58:40.835105 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:40.835151 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260808 22:58:40.898772 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:40.898514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:42949)
0808 22:58:40.898674 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:40.898721 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260808 22:58:41.050796 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:41.050497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:48197)
0808 22:58:41.050706 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:41.050747 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":118,"threads_started":1}
I20260808 22:58:41.836140 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:41.835882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:41379)
0808 22:58:41.836039 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:41.836090 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260808 22:58:41.899617 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:41.899333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:47897)
0808 22:58:41.899510 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:41.899564 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260808 22:58:42.051564 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:42.051313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:56349)
0808 22:58:42.051470 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:42.051512 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260808 22:58:42.837047 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:42.836715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:55261)
0808 22:58:42.836899 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:42.836962 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260808 22:58:42.900460 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:42.900195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:40819)
0808 22:58:42.900363 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:42.900410 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260808 22:58:43.052246 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:43.052022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:45613)
0808 22:58:43.052163 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:43.052197 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260808 22:58:43.216756 20111 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:43.216925 20111 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:43.217154 20111 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
W20260808 22:58:43.217658 19681 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
W20260808 22:58:43.217754 19678 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:43.217823 19678 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb377a6b1b3542f1bb65542f4af81cee; no voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6
I20260808 22:58:43.217924 20111 raft_consensus.cc:2726] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:58:43.837913 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:43.837668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:43661)
0808 22:58:43.837807 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:43.837859 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 22:58:43.901288 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:43.901041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:54197)
0808 22:58:43.901191 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:43.901236 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260808 22:58:44.053237 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:44.052934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:36659)
0808 22:58:44.053116 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:44.053157 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260808 22:58:44.838764 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:44.838479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:52587)
0808 22:58:44.838663 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:44.838714 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":125,"threads_started":1}
I20260808 22:58:44.902113 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:44.901857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:59773)
0808 22:58:44.902013 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:44.902060 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260808 22:58:45.054049 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:45.053743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:54301)
0808 22:58:45.053931 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:45.053994 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260808 22:58:45.839641 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:45.839377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:57869)
0808 22:58:45.839538 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:45.839587 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260808 22:58:45.846652 20120 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:45.846753 20120 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:45.847038 20120 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
W20260808 22:58:45.847489 19681 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
W20260808 22:58:45.847558 19678 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:45.847590 19678 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb377a6b1b3542f1bb65542f4af81cee; no voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6
I20260808 22:58:45.847681 20120 raft_consensus.cc:2726] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:58:45.902901 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:45.902659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:37349)
0808 22:58:45.902801 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:45.902850 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260808 22:58:46.054992 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:46.054695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:35539)
0808 22:58:46.054887 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:46.054940 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260808 22:58:46.840435 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:46.840198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:33203)
0808 22:58:46.840339 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:46.840375 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260808 22:58:46.903770 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:46.903446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:56931)
0808 22:58:46.903663 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:46.903719 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":70,"threads_started":1}
I20260808 22:58:47.055857 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:47.055597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:35089)
0808 22:58:47.055757 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:47.055805 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260808 22:58:47.841187 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:47.840926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:44571)
0808 22:58:47.841095 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:47.841136 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260808 22:58:47.904531 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:47.904232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:48801)
0808 22:58:47.904429 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:47.904479 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260808 22:58:48.056638 20128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:48.056359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:57137)
0808 22:58:48.056527 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:48.056574 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":69,"threads_started":1}
I20260808 22:58:48.842072 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:48.841756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:39087)
0808 22:58:48.841933 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:48.841983 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260808 22:58:48.905431 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:48.905172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:59639)
0808 22:58:48.905318 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:48.905369 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260808 22:58:49.057404 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:49.057129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:34091)
0808 22:58:49.057302 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:49.057348 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260808 22:58:49.843150 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:49.842881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:57359)
0808 22:58:49.843051 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:49.843099 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260808 22:58:49.906260 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:49.906004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:42829)
0808 22:58:49.906155 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:49.906207 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260808 22:58:50.058135 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:50.057891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:40627)
0808 22:58:50.058045 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:50.058081 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260808 22:58:50.844086 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:50.843779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:53483)
0808 22:58:50.843955 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:50.844021 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260808 22:58:50.907145 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:50.906887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:42843)
0808 22:58:50.907046 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:50.907093 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260808 22:58:51.058944 20137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:51.058707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:38997)
0808 22:58:51.058846 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:51.058893 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260808 22:58:51.735282 20138 raft_consensus.cc:482] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:51.735402 20138 raft_consensus.cc:504] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae7a4b94d93c461bb39c7bb01ca141c2" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 46721 } } peers { permanent_uuid: "df42cfed9b914849942b0fd3bb89c8b6" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 32841 } } peers { permanent_uuid: "bb377a6b1b3542f1bb65542f4af81cee" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 39399 } }
I20260808 22:58:51.735606 20138 leader_election.cc:288] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721), df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841)
W20260808 22:58:51.736078 19678 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae7a4b94d93c461bb39c7bb01ca141c2 (127.1.212.3:46721): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
W20260808 22:58:51.736176 19681 leader_election.cc:341] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df42cfed9b914849942b0fd3bb89c8b6 (127.1.212.2:32841): Not found: Tablet not found: 5f65abf244b441a3be37143f7a131a39
I20260808 22:58:51.736271 19681 leader_election.cc:302] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb377a6b1b3542f1bb65542f4af81cee; no voters: ae7a4b94d93c461bb39c7bb01ca141c2, df42cfed9b914849942b0fd3bb89c8b6
I20260808 22:58:51.736375 20138 raft_consensus.cc:2726] T 5f65abf244b441a3be37143f7a131a39 P bb377a6b1b3542f1bb65542f4af81cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:58:51.845270 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:51.844978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.1:51723)
0808 22:58:51.845154 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:51.845205 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260808 22:58:51.907964 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:51.907719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.2:49225)
0808 22:58:51.907869 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:51.907915 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260808 22:58:52.059863 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:52.059608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33879 (local address 127.1.212.3:56015)
0808 22:58:52.059765 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:52.059814 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.62:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260808 22:58:52.060901 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19624
I20260808 22:58:52.069538 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19805
I20260808 22:58:52.077385 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 19946
2026-08-08T22:58:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58689 ms)
[----------] 30 tests from SecurityITest (243805 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:58:52 dist-test-slave-1x4g krb5kdc[20144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:58:52 dist-test-slave-1x4g krb5kdc[20144](info): set up 2 sockets
Aug 08 22:58:52 dist-test-slave-1x4g krb5kdc[20144](info): commencing operation
krb5kdc: starting...
W20260808 22:58:53.165743 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:58:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:58:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:58:53.304018 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33959
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:33959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:53.391299 20158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:53.391547 20158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:53.391572 20158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:53.392807 20158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:53.392848 20158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:53.392863 20158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:53.392875 20158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:53.392889 20158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:53.394428 20158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:33959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:33959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:53.394671 20158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:53.394865 20158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:53.400612 20163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.400612 20164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.400614 20166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:53.400650 20158 server_base.cc:1034] running on GCE node
I20260808 22:58:53.401208 20158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:53.401564 20158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:53.402786 20158 hybrid_clock.cc:648] HybridClock initialized: now 1786229933402743 us; error 67 us; skew 500 ppm
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:53.405933 20158 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:58:53.406414 20158 webserver.cc:460] Webserver started at http://127.1.212.62:46801/ using document root <none> and password file <none>
I20260808 22:58:53.406647 20158 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:53.406697 20158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:53.406796 20158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:53.407944 20158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "cb695b948c4f470eb6049c1e64ae3936"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "4f9ac0c6241ed840d62243d4f1c1e7cf"
server_key_iv: "8d5a368a8349decddbbac169bd2eedf3"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.408289 20158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "cb695b948c4f470eb6049c1e64ae3936"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "4f9ac0c6241ed840d62243d4f1c1e7cf"
server_key_iv: "8d5a368a8349decddbbac169bd2eedf3"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.409865 20158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:58:53.410766 20174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.410997 20158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:58:53.411091 20158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "cb695b948c4f470eb6049c1e64ae3936"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "4f9ac0c6241ed840d62243d4f1c1e7cf"
server_key_iv: "8d5a368a8349decddbbac169bd2eedf3"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.411180 20158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:53.429256 20158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:53.431797 20158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:53.431933 20158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:53.436527 20158 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:33959
I20260808 22:58:53.436564 20228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:33959 every 8 connection(s)
I20260808 22:58:53.437008 20158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:53.438015 20229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:53.440604 20229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: Bootstrap starting.
I20260808 22:58:53.440742 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20158
I20260808 22:58:53.440872 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:58:53.441082 1872 external_mini_cluster.cc:1428] Setting key 65b0eaec0e34f26afc0869fedbebcde5
I20260808 22:58:53.441431 20229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:53.441850 20229 log.cc:824] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:53.442677 20229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: No bootstrap required, opened a new log
I20260808 22:58:53.445091 20229 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } }
I20260808 22:58:53.445283 20229 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:53.445320 20229 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb695b948c4f470eb6049c1e64ae3936, State: Initialized, Role: FOLLOWER
I20260808 22:58:53.445465 20229 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } }
I20260808 22:58:53.445573 20229 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:53.445603 20229 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:53.445675 20229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:53.446404 20229 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } }
I20260808 22:58:53.446534 20229 leader_election.cc:302] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb695b948c4f470eb6049c1e64ae3936; no voters:
I20260808 22:58:53.446723 20229 leader_election.cc:288] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:58:53.446789 20234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:53.446990 20234 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [term 1 LEADER]: Becoming Leader. State: Replica: cb695b948c4f470eb6049c1e64ae3936, State: Running, Role: LEADER
I20260808 22:58:53.447072 20229 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:53.447144 20234 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } }
I20260808 22:58:53.447566 20234 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb695b948c4f470eb6049c1e64ae3936. Latest consensus state: current_term: 1 leader_uuid: "cb695b948c4f470eb6049c1e64ae3936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } } }
I20260808 22:58:53.447676 20235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb695b948c4f470eb6049c1e64ae3936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb695b948c4f470eb6049c1e64ae3936" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 33959 } } }
I20260808 22:58:53.447804 20235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:53.448056 20234 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936 [sys.catalog]: This master's current role is: LEADER
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
W20260808 22:58:53.449208 20252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:58:53.449270 20252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:58:53.449318 20244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:53.449522 20244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:53.452055 20244 catalog_manager.cc:1347] Generated new cluster ID: bc43c36c103e4244961f7ee258f54c06
I20260808 22:58:53.452112 20244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:53.453485 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.441656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.1.212.62:33959)
0808 22:58:53.441867 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:53.441876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:53.441903 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:58:53.442434 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:53.442441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:53.442494 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:53.442627 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:53.443428 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.444364 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.445523 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.446059 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.446153 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.449460 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:53.449465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:53.449465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:53.449481 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:53.451537 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.452005 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.452010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.452010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.452124 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.452378 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.452380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.452380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.452743 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:53.452976 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:53.453066 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 22:58:53.453251 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260808 22:58:53.458885 20244 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:58:53.459663 20244 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:53.463097 20244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: Generated new TSK 0
I20260808 22:58:53.463323 20244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:58:53.496889 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:53.586946 20257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:53.587239 20257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:53.587287 20257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:53.588598 20257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:53.588685 20257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:53.588734 20257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:53.590535 20257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:53.590869 20257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:53.591158 20257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:53.596884 20262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:53.596987 20257 server_base.cc:1034] running on GCE node
W20260808 22:58:53.596884 20263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.596879 20265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:53.597273 20257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:53.597556 20257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:53.598739 20257 hybrid_clock.cc:648] HybridClock initialized: now 1786229933598724 us; error 37 us; skew 500 ppm
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:53.601678 20257 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:58:53.602203 20257 webserver.cc:460] Webserver started at http://127.1.212.1:40513/ using document root <none> and password file <none>
I20260808 22:58:53.602478 20257 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:53.602550 20257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:53.602710 20257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:53.603914 20257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "f396fda58e68463ea5326bf11550e56b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "74ffb42cad72f523893350c06fbf2d58"
server_key_iv: "c5ff291f588d3c8dbb2eb12601623f7f"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.604338 20257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "f396fda58e68463ea5326bf11550e56b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "74ffb42cad72f523893350c06fbf2d58"
server_key_iv: "c5ff291f588d3c8dbb2eb12601623f7f"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.605894 20257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:58:53.606741 20273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.606967 20257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:53.607064 20257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "f396fda58e68463ea5326bf11550e56b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "74ffb42cad72f523893350c06fbf2d58"
server_key_iv: "c5ff291f588d3c8dbb2eb12601623f7f"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.607127 20257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:53.629472 20257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:53.632059 20257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:53.632190 20257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:53.632398 20257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:53.632758 20257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:53.632824 20257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.632848 20257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:53.632895 20257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.639649 20257 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:37237
I20260808 22:58:53.639703 20395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:37237 every 8 connection(s)
I20260808 22:58:53.640098 20257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:53.643972 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20257
I20260808 22:58:53.644074 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:58:53.644284 1872 external_mini_cluster.cc:1428] Setting key 5ed59e068758df09a3197aea45950772
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:53.651415 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.641051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:42927 (local address 127.1.212.62:33959)
0808 22:58:53.641149 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:53.641151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:53.641351 (+ 200us) server_negotiation.cc:407] Connection header received
0808 22:58:53.641941 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:53.641943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:53.641958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:53.642009 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:53.643058 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.643588 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.645032 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.645310 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.645389 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.648006 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:53.648008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:53.648009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:53.648018 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:53.649828 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.650316 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.650318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.650318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.650395 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.650677 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.650678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.650678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.650842 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:53.650924 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:53.651255 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 22:58:53.651343 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:58:53.651765 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.640993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33959 (local address 127.1.212.1:42927)
0808 22:58:53.641196 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:53.641251 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:53.641820 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:53.642090 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:53.642094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:53.642714 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:53.642927 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.642942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.643696 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.643697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.644922 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.644926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.645401 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.645402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.645536 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.646139 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:53.646148 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:53.647892 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:53.649934 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.649943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.649946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.650223 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.650479 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.650480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.650481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.650596 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.650915 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:53.650921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:53.651091 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 22:58:53.651215 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 22:58:53.651432 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260808 22:58:53.652463 20396 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33959
I20260808 22:58:53.652598 20396 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:53.652848 20396 heartbeater.cc:511] Master 127.1.212.62:33959 requested a full tablet report, sending...
I20260808 22:58:53.653510 20191 ts_manager.cc:194] Registered new tserver with Master: f396fda58e68463ea5326bf11550e56b (127.1.212.1:37237)
I20260808 22:58:53.654156 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:42927
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:58:53.676285 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:53.764079 20401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:53.764361 20401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:53.764406 20401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:53.765789 20401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:53.765879 20401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:53.765926 20401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:53.767603 20401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:53.767932 20401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:53.768217 20401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:53.773825 20409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.773825 20406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.773849 20407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:53.774077 20401 server_base.cc:1034] running on GCE node
I20260808 22:58:53.774402 20401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:53.774678 20401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:53.775835 20401 hybrid_clock.cc:648] HybridClock initialized: now 1786229933775818 us; error 48 us; skew 500 ppm
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:53.778698 20401 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:58:53.779213 20401 webserver.cc:460] Webserver started at http://127.1.212.2:41595/ using document root <none> and password file <none>
I20260808 22:58:53.779508 20401 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:53.779585 20401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:53.779749 20401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:53.781000 20401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "038bdf89a4771d54516dab770bfedf84"
server_key_iv: "c6bf0f614f07c91e115d3cdc335c6615"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.781415 20401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "038bdf89a4771d54516dab770bfedf84"
server_key_iv: "c6bf0f614f07c91e115d3cdc335c6615"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.783010 20401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:58:53.783865 20416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.784075 20401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:53.784194 20401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "038bdf89a4771d54516dab770bfedf84"
server_key_iv: "c6bf0f614f07c91e115d3cdc335c6615"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.784281 20401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:53.793692 20401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:53.795883 20401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:53.796064 20401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:53.796329 20401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:53.796698 20401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:53.796758 20401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.796816 20401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:53.796866 20401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.803819 20401 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:39553
I20260808 22:58:53.803905 20536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:39553 every 8 connection(s)
I20260808 22:58:53.804306 20401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:53.812925 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20401
I20260808 22:58:53.813053 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:58:53.813289 1872 external_mini_cluster.cc:1428] Setting key 29a1f5a38e5d377e7b47815d21d4f5ae
I20260808 22:58:53.814837 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.805263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:60611 (local address 127.1.212.62:33959)
0808 22:58:53.805311 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:53.805313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:53.805531 (+ 218us) server_negotiation.cc:407] Connection header received
0808 22:58:53.806101 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:53.806103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:53.806118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:53.806185 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:53.807248 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.807768 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.808850 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.809071 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.809166 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.811645 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:53.811647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:53.811647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:53.811656 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:53.813144 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.813659 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.813661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.813661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.813736 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.814070 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.814072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.814072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.814234 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:53.814297 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:53.814690 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 22:58:53.814771 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 22:58:53.815196 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.805214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33959 (local address 127.1.212.2:60611)
0808 22:58:53.805376 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:53.805439 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:53.805981 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:53.806265 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:53.806270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:53.806921 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:53.807137 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.807142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.807881 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.807882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.808722 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.808725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.809182 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.809183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.809322 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.809851 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:53.809861 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:53.811530 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:53.813231 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.813241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.813244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.813566 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.813815 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.813816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.813816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.813985 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.814316 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:53.814326 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:53.814534 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 22:58:53.814666 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 22:58:53.814872 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260808 22:58:53.815776 20537 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33959
I20260808 22:58:53.815898 20537 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:53.816140 20537 heartbeater.cc:511] Master 127.1.212.62:33959 requested a full tablet report, sending...
I20260808 22:58:53.816669 20191 ts_manager.cc:194] Registered new tserver with Master: 37fbb6cdffe64659bd3dc6ccb6fd113d (127.1.212.2:39553)
I20260808 22:58:53.817250 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:60611
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:58:53.844986 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:53.935849 20545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:53.936100 20545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:53.936128 20545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:53.937472 20545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:53.937530 20545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:53.937561 20545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:53.939138 20545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:53.939369 20545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:53.939589 20545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:53.945152 20550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.945147 20551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:53.945147 20553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:53.945334 20545 server_base.cc:1034] running on GCE node
I20260808 22:58:53.945668 20545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:53.945914 20545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:53.947134 20545 hybrid_clock.cc:648] HybridClock initialized: now 1786229933947119 us; error 30 us; skew 500 ppm
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:53.950256 20545 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:58:53.950700 20545 webserver.cc:460] Webserver started at http://127.1.212.3:42715/ using document root <none> and password file <none>
I20260808 22:58:53.950917 20545 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:53.950960 20545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:53.951058 20545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:53.952178 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "3068c6b6421546b3b065f164c1bf706b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "848e511e2485b5ac9f377721736e5f95"
server_key_iv: "ac11e1b26ae46ef8cba15f7a98152398"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.952524 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "3068c6b6421546b3b065f164c1bf706b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "848e511e2485b5ac9f377721736e5f95"
server_key_iv: "ac11e1b26ae46ef8cba15f7a98152398"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.954223 20545 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:58:53.955003 20560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.955201 20545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:53.955296 20545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "3068c6b6421546b3b065f164c1bf706b"
format_stamp: "Formatted at 2026-08-08 22:58:53 on dist-test-slave-1x4g"
server_key: "848e511e2485b5ac9f377721736e5f95"
server_key_iv: "ac11e1b26ae46ef8cba15f7a98152398"
server_key_version: "encryptionkey@0"
I20260808 22:58:53.955361 20545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:53.964313 20545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:53.966588 20545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:53.966712 20545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:53.966904 20545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:53.967187 20545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:53.967221 20545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.967243 20545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:53.967257 20545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:53.973829 20545 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:35835
I20260808 22:58:53.973909 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:35835 every 8 connection(s)
I20260808 22:58:53.974257 20545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:58:53 dist-test-slave-1x4g krb5kdc[20144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:53.981997 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20545
I20260808 22:58:53.982123 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:58:53.982347 1872 external_mini_cluster.cc:1428] Setting key aea47b340eaf9f86b51d5d0b594475bf
I20260808 22:58:53.984530 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.975065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:47143 (local address 127.1.212.62:33959)
0808 22:58:53.975133 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:53.975135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:53.975377 (+ 242us) server_negotiation.cc:407] Connection header received
0808 22:58:53.975942 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:53.975944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:53.975959 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:53.976012 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:53.977162 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.977688 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.978741 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.978950 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.979028 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.981389 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:53.981391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:53.981391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:53.981407 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:53.983156 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.983675 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.983677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.983677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.983741 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:53.983981 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:53.983982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:53.983982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:53.984102 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:53.984157 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:53.984383 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 22:58:53.984463 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:58:53.984714 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.975049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:33959 (local address 127.1.212.3:47143)
0808 22:58:53.975249 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:53.975288 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:53.975821 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:53.976096 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:53.976099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:53.976721 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:53.977004 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.977009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.977796 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.977797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.978640 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:53.978642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.979052 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.979054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:53.979201 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.979743 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:53.979753 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:53.981272 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:53.983238 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.983247 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.983249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.983585 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.983811 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:53.983812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:53.983812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:53.983922 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:53.984146 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:53.984149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:53.984274 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 22:58:53.984355 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:58:53.984489 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260808 22:58:53.985239 20680 heartbeater.cc:347] Connected to a master server at 127.1.212.62:33959
I20260808 22:58:53.985332 20680 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:53.985486 20680 heartbeater.cc:511] Master 127.1.212.62:33959 requested a full tablet report, sending...
I20260808 22:58:53.985823 20191 ts_manager.cc:194] Registered new tserver with Master: 3068c6b6421546b3b065f164c1bf706b (127.1.212.3:35835)
I20260808 22:58:53.986227 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:47143
I20260808 22:58:53.994259 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:58:54.001864 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:53.995615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.1.212.62:33959)
0808 22:58:53.995683 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:53.995685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:53.995847 (+ 162us) server_negotiation.cc:407] Connection header received
0808 22:58:53.995889 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:53.995889 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:53.995899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:53.995948 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:53.996486 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.997070 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.997785 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:53.997966 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:53.998015 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:53.999161 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:53.999162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:53.999162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:53.999169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:54.000644 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:54.001009 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:54.001010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:54.001010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:54.001064 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:54.001274 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:54.001276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:54.001276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:54.001510 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:54.001603 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.001694 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.001763 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:58:54.003463 20191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:58:54.005215 20191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:58:54.012420 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.008166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.1.212.1:37237)
0808 22:58:54.008365 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.008370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.008381 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:58:54.008425 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.008427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.008443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.008531 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:54.008832 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.009800 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.011461 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.011963 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012065 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012139 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.012252 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.012325 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260808 22:58:54.012430 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.007697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:39553 (local address 127.0.0.1:43932)
0808 22:58:54.008093 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.008106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.008157 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.008853 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.008853 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.008855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:54.008890 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.008891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.010121 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.010122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.011166 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.011169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.012121 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.012189 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012194 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.012224 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.012310 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":104,"threads_started":1}
I20260808 22:58:54.012542 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.007644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:35835 (local address 127.0.0.1:57424)
0808 22:58:54.007787 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.007834 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.007908 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.008506 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.008510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.008513 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:54.008664 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.008667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.010092 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.010093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.011313 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.011316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.012271 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.012325 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012329 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.012379 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.012420 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260808 22:58:54.012423 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.007900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37237 (local address 127.0.0.1:37390)
0808 22:58:54.008141 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.008150 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.008196 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.008618 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.008618 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.008620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:54.008666 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.008667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.009900 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.009901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.011371 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.011373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.012066 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.012152 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012167 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.012237 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.012322 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":91,"thread_start_us":83,"threads_started":1}
I20260808 22:58:54.012887 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.007862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.1.212.3:35835)
0808 22:58:54.008336 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.008341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.008353 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:58:54.008399 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.008401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.008416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.008501 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:54.008730 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.009967 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.011410 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.012108 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012172 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012235 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.012482 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.012531 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":106,"threads_started":1}
I20260808 22:58:54.012945 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.007894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.1.212.2:39553)
0808 22:58:54.008671 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.008675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.008685 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:58:54.008725 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.008726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.008741 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.008857 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:54.008942 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.009999 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.011260 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.012008 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.012668 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.012745 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.012798 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.012849 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":80,"threads_started":1}
I20260808 22:58:54.013731 20469 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa65c9711cc4974a42099d72281abe4 (DEFAULT_TABLE table=test-table [id=615d1d55d9ca494db95a75c4daa7bc8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:54.013893 20613 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa65c9711cc4974a42099d72281abe4 (DEFAULT_TABLE table=test-table [id=615d1d55d9ca494db95a75c4daa7bc8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:54.013984 20324 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa65c9711cc4974a42099d72281abe4 (DEFAULT_TABLE table=test-table [id=615d1d55d9ca494db95a75c4daa7bc8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:54.014191 20469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa65c9711cc4974a42099d72281abe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:54.014191 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa65c9711cc4974a42099d72281abe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:54.014349 20324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa65c9711cc4974a42099d72281abe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:54.017278 20699 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Bootstrap starting.
I20260808 22:58:54.017782 20700 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Bootstrap starting.
I20260808 22:58:54.017992 20699 tablet_bootstrap.cc:654] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:54.018121 20701 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Bootstrap starting.
I20260808 22:58:54.018527 20700 tablet_bootstrap.cc:654] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:54.018782 20701 tablet_bootstrap.cc:654] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:54.018829 20699 log.cc:824] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:54.019196 20700 log.cc:824] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:54.019418 20701 log.cc:824] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:54.019866 20699 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: No bootstrap required, opened a new log
I20260808 22:58:54.019958 20700 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: No bootstrap required, opened a new log
I20260808 22:58:54.019977 20699 ts_tablet_manager.cc:1397] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:58:54.020047 20700 ts_tablet_manager.cc:1397] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:58:54.020334 20701 tablet_bootstrap.cc:492] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: No bootstrap required, opened a new log
I20260808 22:58:54.020408 20701 ts_tablet_manager.cc:1397] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:58:54.021845 20699 raft_consensus.cc:348] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022034 20699 raft_consensus.cc:374] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:54.022081 20699 raft_consensus.cc:729] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f396fda58e68463ea5326bf11550e56b, State: Initialized, Role: FOLLOWER
I20260808 22:58:54.022212 20699 consensus_queue.cc:260] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022224 20700 raft_consensus.cc:348] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022423 20700 raft_consensus.cc:374] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:54.022482 20700 raft_consensus.cc:729] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fbb6cdffe64659bd3dc6ccb6fd113d, State: Initialized, Role: FOLLOWER
I20260808 22:58:54.022461 20701 raft_consensus.cc:348] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022589 20699 ts_tablet_manager.cc:1428] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:54.022641 20701 raft_consensus.cc:374] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:54.022681 20701 raft_consensus.cc:729] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3068c6b6421546b3b065f164c1bf706b, State: Initialized, Role: FOLLOWER
I20260808 22:58:54.022637 20700 consensus_queue.cc:260] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022758 20396 heartbeater.cc:503] Master 127.1.212.62:33959 was elected leader, sending a full tablet report...
I20260808 22:58:54.022796 20701 consensus_queue.cc:260] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.022941 20700 ts_tablet_manager.cc:1428] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:54.022950 20537 heartbeater.cc:503] Master 127.1.212.62:33959 was elected leader, sending a full tablet report...
I20260808 22:58:54.023120 20701 ts_tablet_manager.cc:1428] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:54.023231 20680 heartbeater.cc:503] Master 127.1.212.62:33959 was elected leader, sending a full tablet report...
W20260808 22:58:54.055071 20538 tablet.cc:2369] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:58:54.140965 20397 tablet.cc:2369] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:58:54.225075 20681 tablet.cc:2369] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:54.309674 20708 raft_consensus.cc:482] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:54.309877 20708 raft_consensus.cc:504] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.310302 20708 leader_election.cc:288] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3068c6b6421546b3b065f164c1bf706b (127.1.212.3:35835), 37fbb6cdffe64659bd3dc6ccb6fd113d (127.1.212.2:39553)
I20260808 22:58:54.314836 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.310733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:48807 (local address 127.1.212.3:35835)
0808 22:58:54.310784 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.310787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.310887 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:58:54.310954 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.310955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.310969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.311052 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:54.311351 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.312126 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.313807 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.314420 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.314493 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.314506 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.314701 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.314762 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:58:54.314836 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.310841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:41425 (local address 127.1.212.2:39553)
0808 22:58:54.310919 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.310921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.310932 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:58:54.310957 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.310958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.310970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.311051 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:54.311351 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.312113 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.313807 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.314409 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.314493 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.314505 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.314704 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.314755 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:58:54.314901 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.310631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:35835 (local address 127.1.212.1:48807)
0808 22:58:54.310799 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.310812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.310878 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.311099 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.311100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.311104 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:54.311270 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.311271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.312220 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.312221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.313695 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.313700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.314512 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.314512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.314605 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.314613 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.314677 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.314746 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":91,"thread_start_us":85,"threads_started":1}
I20260808 22:58:54.314905 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.310631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:39553 (local address 127.1.212.1:41425)
0808 22:58:54.310800 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.310812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.310896 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.311099 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.311100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.311104 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:54.311241 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.311248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.312220 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.312221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.313695 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.313700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.314509 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.314510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.314605 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.314612 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.314681 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.314746 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":136,"thread_start_us":67,"threads_started":1}
I20260808 22:58:54.315277 20634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa65c9711cc4974a42099d72281abe4" candidate_uuid: "f396fda58e68463ea5326bf11550e56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3068c6b6421546b3b065f164c1bf706b" is_pre_election: true
I20260808 22:58:54.315531 20634 raft_consensus.cc:2454] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f396fda58e68463ea5326bf11550e56b in term 0.
I20260808 22:58:54.315511 20489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa65c9711cc4974a42099d72281abe4" candidate_uuid: "f396fda58e68463ea5326bf11550e56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" is_pre_election: true
I20260808 22:58:54.315649 20489 raft_consensus.cc:2454] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f396fda58e68463ea5326bf11550e56b in term 0.
I20260808 22:58:54.315827 20274 leader_election.cc:302] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3068c6b6421546b3b065f164c1bf706b, f396fda58e68463ea5326bf11550e56b; no voters:
I20260808 22:58:54.316009 20708 raft_consensus.cc:2781] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:58:54.316083 20708 raft_consensus.cc:482] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:58:54.316110 20708 raft_consensus.cc:3037] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:54.317054 20708 raft_consensus.cc:504] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.317237 20708 leader_election.cc:288] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [CANDIDATE]: Term 1 election: Requested vote from peers 3068c6b6421546b3b065f164c1bf706b (127.1.212.3:35835), 37fbb6cdffe64659bd3dc6ccb6fd113d (127.1.212.2:39553)
I20260808 22:58:54.317487 20634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa65c9711cc4974a42099d72281abe4" candidate_uuid: "f396fda58e68463ea5326bf11550e56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3068c6b6421546b3b065f164c1bf706b"
I20260808 22:58:54.317546 20489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa65c9711cc4974a42099d72281abe4" candidate_uuid: "f396fda58e68463ea5326bf11550e56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d"
I20260808 22:58:54.317620 20489 raft_consensus.cc:3037] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:54.317607 20634 raft_consensus.cc:3037] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:54.318449 20634 raft_consensus.cc:2454] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f396fda58e68463ea5326bf11550e56b in term 1.
I20260808 22:58:54.318482 20489 raft_consensus.cc:2454] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f396fda58e68463ea5326bf11550e56b in term 1.
I20260808 22:58:54.318672 20274 leader_election.cc:302] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3068c6b6421546b3b065f164c1bf706b, f396fda58e68463ea5326bf11550e56b; no voters:
I20260808 22:58:54.318792 20708 raft_consensus.cc:2781] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:54.318961 20708 raft_consensus.cc:686] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 1 LEADER]: Becoming Leader. State: Replica: f396fda58e68463ea5326bf11550e56b, State: Running, Role: LEADER
I20260808 22:58:54.319077 20708 consensus_queue.cc:237] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } }
I20260808 22:58:54.320065 20190 catalog_manager.cc:5495] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b reported cstate change: term changed from 0 to 1, leader changed from <none> to f396fda58e68463ea5326bf11550e56b (127.1.212.1). New cstate: current_term: 1 leader_uuid: "f396fda58e68463ea5326bf11550e56b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f396fda58e68463ea5326bf11550e56b" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37237 } health_report { overall_health: HEALTHY } } }
I20260808 22:58:54.368041 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.365653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.1.212.1:37237)
0808 22:58:54.365728 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.365730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.365739 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:58:54.365783 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.365783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.365807 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.365880 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:54.366071 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.366559 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.367349 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.367548 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.367601 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.367746 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:54.367860 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:58:54.367923 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.367952 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.367975 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:58:54.370296 20634 raft_consensus.cc:1261] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 1 FOLLOWER]: Refusing update from remote peer f396fda58e68463ea5326bf11550e56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:54.370296 20489 raft_consensus.cc:1261] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 1 FOLLOWER]: Refusing update from remote peer f396fda58e68463ea5326bf11550e56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:54.370746 20708 consensus_queue.cc:1035] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fbb6cdffe64659bd3dc6ccb6fd113d" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:54.370923 20713 consensus_queue.cc:1035] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3068c6b6421546b3b065f164c1bf706b" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 35835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:54.373663 20717 mvcc.cc:205] Tried to move back new op lower bound from 7316397811177136128 to 7316397810971656192. Current Snapshot: MvccSnapshot[applied={T|T < 7316397811177136128}]
I20260808 22:58:54.376686 20190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:54.376838 20190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:54.377674 20190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb695b948c4f470eb6049c1e64ae3936: Sending DeleteTablet for 3 replicas of tablet 1aa65c9711cc4974a42099d72281abe4
I20260808 22:58:54.378010 20613 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa65c9711cc4974a42099d72281abe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:57424
I20260808 22:58:54.378051 20469 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa65c9711cc4974a42099d72281abe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:43932
I20260808 22:58:54.378134 20324 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa65c9711cc4974a42099d72281abe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:54 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:37390
I20260808 22:58:54.378345 20728 tablet_replica.cc:330] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b: stopping tablet replica
I20260808 22:58:54.378487 20726 tablet_replica.cc:330] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: stopping tablet replica
I20260808 22:58:54.378571 20728 raft_consensus.cc:2229] T 1aa65c9711cc4974a42099d72281abe4 P f396fda58e68463ea5326bf11550e56b [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:58:54.378648 20726 raft_consensus.cc:2229] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:54.378655 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20257
I20260808 22:58:54.381724 20726 raft_consensus.cc:2258] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:54.381986 20727 tablet_replica.cc:330] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: stopping tablet replica
I20260808 22:58:54.382128 20727 raft_consensus.cc:2229] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:54.382252 20727 raft_consensus.cc:2258] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:54.382400 20726 ts_tablet_manager.cc:1905] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:54.382725 20727 ts_tablet_manager.cc:1905] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:54.384569 20727 ts_tablet_manager.cc:1918] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:54.384683 20727 log.cc:1196] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/1aa65c9711cc4974a42099d72281abe4
I20260808 22:58:54.384690 20726 ts_tablet_manager.cc:1918] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:54.384835 20726 log.cc:1196] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/1aa65c9711cc4974a42099d72281abe4
I20260808 22:58:54.385020 20727 ts_tablet_manager.cc:1939] T 1aa65c9711cc4974a42099d72281abe4 P 37fbb6cdffe64659bd3dc6ccb6fd113d: Deleting consensus metadata
I20260808 22:58:54.385143 20726 ts_tablet_manager.cc:1939] T 1aa65c9711cc4974a42099d72281abe4 P 3068c6b6421546b3b065f164c1bf706b: Deleting consensus metadata
I20260808 22:58:54.385571 20177 catalog_manager.cc:4841] TS 37fbb6cdffe64659bd3dc6ccb6fd113d (127.1.212.2:39553): tablet 1aa65c9711cc4974a42099d72281abe4 (table test-table [id=615d1d55d9ca494db95a75c4daa7bc8e]) successfully deleted
I20260808 22:58:54.385628 20175 catalog_manager.cc:4841] TS 3068c6b6421546b3b065f164c1bf706b (127.1.212.3:35835): tablet 1aa65c9711cc4974a42099d72281abe4 (table test-table [id=615d1d55d9ca494db95a75c4daa7bc8e]) successfully deleted
W20260808 22:58:54.386723 20177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:37237 (error 108)
I20260808 22:58:54.386957 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20401
I20260808 22:58:54.388616 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.388473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37237 (local address 127.0.0.1:37404)
0808 22:58:54.388521 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.388570 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 22:58:54.388797 20177 catalog_manager.cc:4584] TS f396fda58e68463ea5326bf11550e56b (127.1.212.1:37237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1aa65c9711cc4974a42099d72281abe4: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37237: connect: Connection refused (error 111)
I20260808 22:58:54.398398 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20545
I20260808 22:58:54.405682 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20158
2026-08-08T22:58:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2328 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T22:58:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:58:54Z Disabled control of system clock
I20260808 22:58:54.439463 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37393
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:54.526860 20733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:54.527118 20733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:54.527141 20733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:54.528311 20733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:54.528352 20733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:54.528365 20733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:54.528378 20733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:54.528389 20733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:54.529803 20733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37393
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:54.530287 20733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:54.530602 20733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:54.536139 20742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.536155 20738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.536185 20733 server_base.cc:1034] running on GCE node
W20260808 22:58:54.536163 20739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.536732 20733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:54.537122 20733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:54.538302 20733 hybrid_clock.cc:648] HybridClock initialized: now 1786229934538275 us; error 41 us; skew 500 ppm
I20260808 22:58:54.539786 20733 webserver.cc:460] Webserver started at http://127.1.212.62:42963/ using document root <none> and password file <none>
I20260808 22:58:54.540063 20733 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:54.540120 20733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:54.540304 20733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:54.541539 20733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "989453b49fb2498ea7f76e79ca6c674f"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "e69c5cedddd65f0a2c592b8e9fb599e3"
server_key_iv: "7df3782644b6b209bfcd5a33766ad776"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.541949 20733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "989453b49fb2498ea7f76e79ca6c674f"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "e69c5cedddd65f0a2c592b8e9fb599e3"
server_key_iv: "7df3782644b6b209bfcd5a33766ad776"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.543304 20733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:58:54.544080 20748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.544283 20733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:54.544494 20733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "989453b49fb2498ea7f76e79ca6c674f"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "e69c5cedddd65f0a2c592b8e9fb599e3"
server_key_iv: "7df3782644b6b209bfcd5a33766ad776"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.544600 20733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:54.559324 20733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:54.559716 20733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:54.559894 20733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:54.564465 20733 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37393
I20260808 22:58:54.564504 20802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37393 every 8 connection(s)
I20260808 22:58:54.564981 20733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:54.565243 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20733
I20260808 22:58:54.565361 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:58:54.565565 1872 external_mini_cluster.cc:1428] Setting key ccb676c7f7fc7520067301a4b59fb3c9
I20260808 22:58:54.566359 20803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:54.569873 20803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: Bootstrap starting.
I20260808 22:58:54.570650 20803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:54.571015 20803 log.cc:824] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:54.571938 20803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: No bootstrap required, opened a new log
I20260808 22:58:54.572137 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.566167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51398 (local address 127.1.212.62:37393)
0808 22:58:54.566425 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.566435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.566471 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:58:54.567114 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.567124 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.567220 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.567395 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:54.567568 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.569008 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.569836 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.570456 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.570605 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.571342 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:54.571347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:54.571347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:54.571529 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:54.571599 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:54.571615 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:54.571684 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.571766 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.571852 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260808 22:58:54.574100 20803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } }
I20260808 22:58:54.574255 20803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:54.574280 20803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989453b49fb2498ea7f76e79ca6c674f, State: Initialized, Role: FOLLOWER
I20260808 22:58:54.574404 20803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } }
I20260808 22:58:54.574487 20803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:54.574512 20803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:54.574541 20803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:54.575127 20803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } }
I20260808 22:58:54.575219 20803 leader_election.cc:302] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989453b49fb2498ea7f76e79ca6c674f; no voters:
I20260808 22:58:54.575421 20803 leader_election.cc:288] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:58:54.575462 20809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:54.575603 20809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [term 1 LEADER]: Becoming Leader. State: Replica: 989453b49fb2498ea7f76e79ca6c674f, State: Running, Role: LEADER
I20260808 22:58:54.575752 20803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:54.575769 20809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } }
I20260808 22:58:54.576170 20811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989453b49fb2498ea7f76e79ca6c674f. Latest consensus state: current_term: 1 leader_uuid: "989453b49fb2498ea7f76e79ca6c674f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } } }
I20260808 22:58:54.576270 20811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:54.576484 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989453b49fb2498ea7f76e79ca6c674f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989453b49fb2498ea7f76e79ca6c674f" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37393 } } }
I20260808 22:58:54.576644 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:54.577112 20818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:54.577752 20818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:54.579468 20818 catalog_manager.cc:1347] Generated new cluster ID: 69155db3a4e64a97bcec3baa715865f0
I20260808 22:58:54.579526 20818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:54.591142 20818 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:58:54.591725 20818 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:54.599349 20818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: Generated new TSK 0
I20260808 22:58:54.599634 20818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:58:54.606254 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--builtin_ntp_servers=127.1.212.20:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:54.697729 20831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:54.697968 20831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:54.697992 20831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:54.699316 20831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:54.699373 20831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:54.699402 20831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:54.700923 20831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:54.701215 20831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:54.701562 20831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:54.707087 20836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.707207 20837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.707087 20839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.708247 20831 server_base.cc:1034] running on GCE node
I20260808 22:58:54.708498 20831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:54.708807 20831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:54.709998 20831 hybrid_clock.cc:648] HybridClock initialized: now 1786229934709985 us; error 34 us; skew 500 ppm
I20260808 22:58:54.711500 20831 webserver.cc:460] Webserver started at http://127.1.212.1:37559/ using document root <none> and password file <none>
I20260808 22:58:54.711747 20831 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:54.711794 20831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:54.711895 20831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:54.713135 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "7b7614c157b34a33adc7552a2706e00c"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "1d7acf8848cbdca17613c7587544b9e9"
server_key_iv: "e482dc130127faaaafb370a4e10f78b0"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.713476 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "7b7614c157b34a33adc7552a2706e00c"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "1d7acf8848cbdca17613c7587544b9e9"
server_key_iv: "e482dc130127faaaafb370a4e10f78b0"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.714891 20831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:58:54.715773 20845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.716027 20831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:54.716114 20831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "7b7614c157b34a33adc7552a2706e00c"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "1d7acf8848cbdca17613c7587544b9e9"
server_key_iv: "e482dc130127faaaafb370a4e10f78b0"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.716221 20831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:54.735440 20831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:54.735839 20831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:54.736020 20831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:54.736302 20831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:54.736728 20831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:54.736766 20831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.736907 20831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:54.736953 20831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.743521 20831 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:35033
I20260808 22:58:54.743613 20965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:35033 every 8 connection(s)
I20260808 22:58:54.744035 20831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:54.750209 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.744946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:58333 (local address 127.1.212.62:37393)
0808 22:58:54.744994 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.744996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.745290 (+ 294us) server_negotiation.cc:407] Connection header received
0808 22:58:54.745887 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.745889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.745910 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.745980 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:54.746759 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.747373 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.748544 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.748899 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.748982 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.749873 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:54.749874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:54.749875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:54.749881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:54.749898 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:54.749902 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:54.749956 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.750069 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.750119 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:58:54.750453 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.744941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37393 (local address 127.1.212.1:58333)
0808 22:58:54.745151 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.745189 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.745763 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.746061 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.746064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.746074 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:54.746651 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.746657 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.747491 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.747492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.748444 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.748448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.748999 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.749000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.749153 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.749717 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:54.749718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:54.749750 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:54.749778 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:54.749967 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:54.749973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:54.749979 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.750058 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.750178 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260808 22:58:54.751025 20966 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37393
I20260808 22:58:54.751363 20966 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:54.751797 20966 heartbeater.cc:511] Master 127.1.212.62:37393 requested a full tablet report, sending...
I20260808 22:58:54.752455 20767 ts_manager.cc:194] Registered new tserver with Master: 7b7614c157b34a33adc7552a2706e00c (127.1.212.1:35033)
I20260808 22:58:54.753154 20767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.1:58333
I20260808 22:58:54.753489 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20831
I20260808 22:58:54.753608 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:58:54.753842 1872 external_mini_cluster.cc:1428] Setting key 3750e5a262e1f68b5c39ed725f6e93c3
I20260808 22:58:54.755050 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--builtin_ntp_servers=127.1.212.20:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:54.847589 20969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:54.847886 20969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:54.847929 20969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:54.849319 20969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:54.849390 20969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:54.849444 20969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:54.851038 20969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:54.851327 20969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:54.851665 20969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:54.857089 20978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.857095 20976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.857126 20969 server_base.cc:1034] running on GCE node
W20260808 22:58:54.857128 20974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.857553 20969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:54.857853 20969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:54.859021 20969 hybrid_clock.cc:648] HybridClock initialized: now 1786229934859012 us; error 56 us; skew 500 ppm
I20260808 22:58:54.860846 20969 webserver.cc:460] Webserver started at http://127.1.212.2:32961/ using document root <none> and password file <none>
I20260808 22:58:54.861248 20969 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:54.861321 20969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:54.861485 20969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:54.862848 20969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "585b15bd05374bfa8917433c78291e28"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "33135fc07065a0b6c6dae91a0763a682"
server_key_iv: "322174b60475ce6c444eadd9e7f4a56b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.863274 20969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "585b15bd05374bfa8917433c78291e28"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "33135fc07065a0b6c6dae91a0763a682"
server_key_iv: "322174b60475ce6c444eadd9e7f4a56b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.864755 20969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 22:58:54.865725 20984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.865943 20969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:54.866060 20969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "585b15bd05374bfa8917433c78291e28"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "33135fc07065a0b6c6dae91a0763a682"
server_key_iv: "322174b60475ce6c444eadd9e7f4a56b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.866158 20969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:54.878706 20969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:54.879107 20969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:54.879285 20969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:54.879558 20969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:54.879930 20969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:54.879997 20969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.880035 20969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:54.880084 20969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.886358 20969 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44453
I20260808 22:58:54.886407 21108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44453 every 8 connection(s)
I20260808 22:58:54.886790 20969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:58:54.891471 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 20969
I20260808 22:58:54.891618 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:58:54.891849 1872 external_mini_cluster.cc:1428] Setting key 193975ea5a4f8a9cecf0c3302d498ca8
I20260808 22:58:54.892776 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.887635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:41459 (local address 127.1.212.62:37393)
0808 22:58:54.887708 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:54.887711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:54.888031 (+ 320us) server_negotiation.cc:407] Connection header received
0808 22:58:54.888569 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:54.888570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:54.888584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:54.888642 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:54.889470 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.889973 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.891039 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.891276 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.891368 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.892425 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:54.892427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:54.892427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:54.892432 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:54.892449 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:54.892453 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:54.892521 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:54.892641 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 22:58:54.892693 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:58:54.893023 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--builtin_ntp_servers=127.1.212.20:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:58:54.893081 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:54.887637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37393 (local address 127.1.212.2:41459)
0808 22:58:54.887893 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:54.887939 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:54.888472 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:54.888731 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:54.888735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:54.888744 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:54.889362 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.889367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.890102 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.890103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.890939 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:54.890941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:54.891386 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:54.891387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:54.891540 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:54.892259 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:54.892260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:54.892302 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:54.892330 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:54.892523 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:54.892530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:54.892533 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:58:54.892627 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:58:54.892759 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
I20260808 22:58:54.895074 21109 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37393
I20260808 22:58:54.895244 21109 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:54.895474 21109 heartbeater.cc:511] Master 127.1.212.62:37393 requested a full tablet report, sending...
I20260808 22:58:54.895989 20767 ts_manager.cc:194] Registered new tserver with Master: 585b15bd05374bfa8917433c78291e28 (127.1.212.2:44453)
I20260808 22:58:54.896392 20767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.2:41459
W20260808 22:58:54.982205 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:54.982440 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:54.982465 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:54.983758 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:54.983815 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:54.983844 21112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:54.985404 21112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:54.985703 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:54.986057 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:54.991626 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.991595 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:54.991583 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:54.991878 21112 server_base.cc:1034] running on GCE node
I20260808 22:58:54.992120 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:54.992419 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:54.993587 21112 hybrid_clock.cc:648] HybridClock initialized: now 1786229934993581 us; error 39 us; skew 500 ppm
I20260808 22:58:54.995034 21112 webserver.cc:460] Webserver started at http://127.1.212.3:41453/ using document root <none> and password file <none>
I20260808 22:58:54.995344 21112 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:54.995421 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:54.995582 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:54.996739 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "fe7986120db744d071faff89008dd882"
server_key_iv: "89fdf7f6a8cbceff964cd9b15bbe667b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.997267 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "fe7986120db744d071faff89008dd882"
server_key_iv: "89fdf7f6a8cbceff964cd9b15bbe667b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.998708 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:58:54.999536 21126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:54.999727 21112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:58:54.999840 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec"
format_stamp: "Formatted at 2026-08-08 22:58:54 on dist-test-slave-1x4g"
server_key: "fe7986120db744d071faff89008dd882"
server_key_iv: "89fdf7f6a8cbceff964cd9b15bbe667b"
server_key_version: "encryptionkey@0"
I20260808 22:58:54.999922 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:55.015763 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:55.016146 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:55.016316 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:55.016607 21112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:55.017059 21112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:55.017129 21112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:55.017177 21112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:55.017205 21112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:55.023888 21112 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:43311
I20260808 22:58:55.023942 21246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:43311 every 8 connection(s)
I20260808 22:58:55.024341 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:58:55.029688 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21112
I20260808 22:58:55.029819 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:58:55.030071 1872 external_mini_cluster.cc:1428] Setting key d453ac38279d6efa5bd0d5a32aa7f2a8
I20260808 22:58:55.030210 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.025198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:58593 (local address 127.1.212.62:37393)
0808 22:58:55.025278 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.025280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.025520 (+ 240us) server_negotiation.cc:407] Connection header received
0808 22:58:55.026058 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.026059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.026071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.026139 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:55.026853 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.027380 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.028476 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.028677 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.028739 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.029808 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:55.029809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:55.029810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:55.029817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:55.029836 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:55.029841 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:55.029921 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.030057 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.030092 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:58:55.030453 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.025191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37393 (local address 127.1.212.3:58593)
0808 22:58:55.025403 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.025444 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.025966 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.026198 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.026201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.026210 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:55.026747 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.026752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.027503 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.027504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.028359 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.028367 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.028783 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.028784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.028914 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.029592 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:55.029593 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:55.029641 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:55.029677 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:55.029941 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:55.029969 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:55.029973 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.030052 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.030189 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260808 22:58:55.031085 21247 heartbeater.cc:347] Connected to a master server at 127.1.212.62:37393
I20260808 22:58:55.031219 21247 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:55.031409 21247 heartbeater.cc:511] Master 127.1.212.62:37393 requested a full tablet report, sending...
I20260808 22:58:55.031796 20767 ts_manager.cc:194] Registered new tserver with Master: 0fb7d8fbd43b4b358ff2d8df617ac1ec (127.1.212.3:43311)
I20260808 22:58:55.032202 20767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.3:58593
I20260808 22:58:55.042131 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:58:55.046662 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.043652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.1.212.62:37393)
0808 22:58:55.043713 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.043715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.043845 (+ 130us) server_negotiation.cc:407] Connection header received
0808 22:58:55.043881 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.043882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.043892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.043949 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:55.044125 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.044633 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.045496 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.045693 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.045762 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.046437 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:55.046439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:55.046439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:55.046444 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:55.046461 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:55.046465 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:55.046516 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.046567 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.046599 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:58:55.048424 20767 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:58:55.050210 20767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:58:55.057153 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.052736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35033 (local address 127.0.0.1:43048)
0808 22:58:55.053170 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.053179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.053218 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.053578 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.053581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.053583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.053662 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.053664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.054628 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.054629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.055851 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.055855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.056817 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.056819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.056909 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.056926 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.057011 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.057082 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":6,"thread_start_us":94,"threads_started":1}
I20260808 22:58:55.057436 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.052736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44453 (local address 127.0.0.1:60586)
0808 22:58:55.052951 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.052994 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.053071 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.053666 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.053667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.053669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.053731 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.053733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.055098 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.055099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.056097 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.056099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.057191 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.057192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.057252 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.057256 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.057308 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.057349 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
I20260808 22:58:55.057689 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.052958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.1.212.2:44453)
0808 22:58:55.053307 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.053311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.053320 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:58:55.053366 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.053368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.053386 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.053464 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.053851 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.055004 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.056194 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.056986 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.057057 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.057140 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.057560 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.057605 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":84,"threads_started":1}
I20260808 22:58:55.058033 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.052959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.1.212.1:35033)
0808 22:58:55.053196 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.053201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.053213 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:58:55.053388 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.053391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.053407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.053484 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.053748 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.054513 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.055953 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.056689 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.056797 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.056931 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.057852 (+ 921us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.057936 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
I20260808 22:58:55.058601 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.052753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43311 (local address 127.0.0.1:36024)
0808 22:58:55.053484 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.053495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.053553 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.053710 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.053711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.053713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.053751 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.053753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.056170 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.056170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.056950 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.056952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.058348 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.058349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.058416 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.058420 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.058480 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.058521 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":1,"thread_start_us":161,"threads_started":1}
I20260808 22:58:55.058629 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.053066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36024 (local address 127.1.212.3:43311)
0808 22:58:55.053273 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.053278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.053558 (+ 280us) server_negotiation.cc:407] Connection header received
0808 22:58:55.053616 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.053618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.053634 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.053715 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.053807 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.054997 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.057025 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.058219 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.058287 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.058392 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.058496 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.058546 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":106,"threads_started":1}
I20260808 22:58:55.058533 21040 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae17df523c4199af5b7a96b418ca45 (DEFAULT_TABLE table=test-table [id=158ae02eb310436aa760895ad25bef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:55.058735 20896 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae17df523c4199af5b7a96b418ca45 (DEFAULT_TABLE table=test-table [id=158ae02eb310436aa760895ad25bef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:55.058959 21040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae17df523c4199af5b7a96b418ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:55.059127 20896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae17df523c4199af5b7a96b418ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:55.059289 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 98ae17df523c4199af5b7a96b418ca45 (DEFAULT_TABLE table=test-table [id=158ae02eb310436aa760895ad25bef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:55.059631 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ae17df523c4199af5b7a96b418ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:55.062556 21262 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Bootstrap starting.
I20260808 22:58:55.063308 21262 tablet_bootstrap.cc:654] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:55.063475 21264 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Bootstrap starting.
I20260808 22:58:55.063619 21263 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Bootstrap starting.
I20260808 22:58:55.063719 21262 log.cc:824] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:55.064229 21264 tablet_bootstrap.cc:654] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:55.064237 21263 tablet_bootstrap.cc:654] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:55.064590 21264 log.cc:824] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:55.064594 21263 log.cc:824] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:55.064909 21262 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: No bootstrap required, opened a new log
I20260808 22:58:55.064996 21262 ts_tablet_manager.cc:1397] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:58:55.065428 21264 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: No bootstrap required, opened a new log
I20260808 22:58:55.065526 21264 ts_tablet_manager.cc:1397] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:58:55.065738 21263 tablet_bootstrap.cc:492] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: No bootstrap required, opened a new log
I20260808 22:58:55.065832 21263 ts_tablet_manager.cc:1397] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:58:55.067528 21264 raft_consensus.cc:348] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.067721 21264 raft_consensus.cc:374] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:55.067770 21264 raft_consensus.cc:729] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb7d8fbd43b4b358ff2d8df617ac1ec, State: Initialized, Role: FOLLOWER
I20260808 22:58:55.067893 21264 consensus_queue.cc:260] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.067912 21262 raft_consensus.cc:348] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.068090 21262 raft_consensus.cc:374] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:55.068141 21262 raft_consensus.cc:729] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585b15bd05374bfa8917433c78291e28, State: Initialized, Role: FOLLOWER
I20260808 22:58:55.068228 21264 ts_tablet_manager.cc:1428] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:55.068264 21262 consensus_queue.cc:260] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.068372 21247 heartbeater.cc:503] Master 127.1.212.62:37393 was elected leader, sending a full tablet report...
I20260808 22:58:55.068547 21262 ts_tablet_manager.cc:1428] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:58:55.068598 21109 heartbeater.cc:503] Master 127.1.212.62:37393 was elected leader, sending a full tablet report...
I20260808 22:58:55.068724 21263 raft_consensus.cc:348] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.068928 21263 raft_consensus.cc:374] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:55.068976 21263 raft_consensus.cc:729] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7614c157b34a33adc7552a2706e00c, State: Initialized, Role: FOLLOWER
I20260808 22:58:55.069118 21263 consensus_queue.cc:260] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.069394 21263 ts_tablet_manager.cc:1428] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260808 22:58:55.069638 20966 heartbeater.cc:503] Master 127.1.212.62:37393 was elected leader, sending a full tablet report...
W20260808 22:58:55.137441 21110 tablet.cc:2369] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:55.234074 21271 raft_consensus.cc:482] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:55.234228 21271 raft_consensus.cc:504] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.234747 21271 leader_election.cc:288] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 585b15bd05374bfa8917433c78291e28 (127.1.212.2:44453), 0fb7d8fbd43b4b358ff2d8df617ac1ec (127.1.212.3:43311)
I20260808 22:58:55.236339 21270 raft_consensus.cc:482] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:55.236476 21270 raft_consensus.cc:504] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.237241 21270 leader_election.cc:288] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b7614c157b34a33adc7552a2706e00c (127.1.212.1:35033), 0fb7d8fbd43b4b358ff2d8df617ac1ec (127.1.212.3:43311)
I20260808 22:58:55.238413 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.234921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44453 (local address 127.1.212.1:56345)
0808 22:58:55.235023 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.235036 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.235083 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.235293 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.235294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.235296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.235381 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.235383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.236042 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.236043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.237326 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.237330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238161 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.238233 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.238237 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.238284 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.238332 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":27}
I20260808 22:58:55.238538 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.235040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:60447 (local address 127.1.212.3:43311)
0808 22:58:55.235129 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.235131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.235259 (+ 128us) server_negotiation.cc:407] Connection header received
0808 22:58:55.235337 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.235338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.235353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.235428 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.235605 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.236088 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.237508 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238122 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238172 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.238182 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.238415 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.238466 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:58:55.238776 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.234936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43311 (local address 127.1.212.1:60447)
0808 22:58:55.235196 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.235204 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.235268 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.235433 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.235433 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.235435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.235529 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.235530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.236211 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.236212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.237394 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.237397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238256 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.238329 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.238333 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.238652 (+ 319us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.238699 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":140,"thread_start_us":71,"threads_started":1}
I20260808 22:58:55.239090 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.235063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56345 (local address 127.1.212.2:44453)
0808 22:58:55.235137 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.235140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.235151 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:58:55.235184 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.235185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.235199 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.235286 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.235463 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.235952 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.237434 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238047 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238110 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.238121 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.238965 (+ 844us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.239018 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:58:55.239108 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "7b7614c157b34a33adc7552a2706e00c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" is_pre_election: true
I20260808 22:58:55.239262 21198 raft_consensus.cc:2454] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b7614c157b34a33adc7552a2706e00c in term 0.
I20260808 22:58:55.239326 21060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "7b7614c157b34a33adc7552a2706e00c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585b15bd05374bfa8917433c78291e28" is_pre_election: true
I20260808 22:58:55.239429 21060 raft_consensus.cc:2454] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b7614c157b34a33adc7552a2706e00c in term 0.
I20260808 22:58:55.239498 20847 leader_election.cc:302] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb7d8fbd43b4b358ff2d8df617ac1ec, 7b7614c157b34a33adc7552a2706e00c; no voters:
I20260808 22:58:55.239758 21271 raft_consensus.cc:2781] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:58:55.239825 21271 raft_consensus.cc:482] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:58:55.239856 21271 raft_consensus.cc:3037] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:55.240528 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.237095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35033 (local address 127.1.212.2:46653)
0808 22:58:55.237190 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.237200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.237237 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.237529 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.237530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.237532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.237600 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.237602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238449 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.239561 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.239564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.240278 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.240279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.240348 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.240352 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.240418 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.240460 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 22:58:55.240557 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.237317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46653 (local address 127.1.212.1:35033)
0808 22:58:55.237374 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.237376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.237385 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:58:55.237408 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.237408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.237419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.237482 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.237678 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238349 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.239655 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.240185 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.240265 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.240275 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.240455 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.240497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:58:55.240854 20918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "585b15bd05374bfa8917433c78291e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b7614c157b34a33adc7552a2706e00c" is_pre_election: true
I20260808 22:58:55.240998 21271 raft_consensus.cc:504] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.241128 20918 raft_consensus.cc:2379] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 585b15bd05374bfa8917433c78291e28 in current term 1: Already voted for candidate 7b7614c157b34a33adc7552a2706e00c in this term.
I20260808 22:58:55.241154 21271 leader_election.cc:288] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [CANDIDATE]: Term 1 election: Requested vote from peers 585b15bd05374bfa8917433c78291e28 (127.1.212.2:44453), 0fb7d8fbd43b4b358ff2d8df617ac1ec (127.1.212.3:43311)
I20260808 22:58:55.241315 21060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "7b7614c157b34a33adc7552a2706e00c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585b15bd05374bfa8917433c78291e28"
I20260808 22:58:55.241330 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "7b7614c157b34a33adc7552a2706e00c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec"
I20260808 22:58:55.241400 21060 raft_consensus.cc:3037] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:55.241420 21198 raft_consensus.cc:3037] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:55.241536 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.237103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43311 (local address 127.1.212.2:46997)
0808 22:58:55.237509 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.237518 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:55.237563 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:55.237941 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:55.237942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:55.237944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:55.238003 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.238005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238981 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.238981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.240069 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:55.240072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.241249 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.241250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:55.241337 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.241341 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:55.241398 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:58:55.241446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260808 22:58:55.241807 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.237651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:46997 (local address 127.1.212.3:43311)
0808 22:58:55.237800 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.237802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.237810 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:58:55.237832 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.237833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.237842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.238068 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:55.238105 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.238879 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.240170 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.240995 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.241103 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.241115 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.241693 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.241742 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260808 22:58:55.242012 21197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ae17df523c4199af5b7a96b418ca45" candidate_uuid: "585b15bd05374bfa8917433c78291e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" is_pre_election: true
I20260808 22:58:55.242319 20985 leader_election.cc:302] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 585b15bd05374bfa8917433c78291e28; no voters: 0fb7d8fbd43b4b358ff2d8df617ac1ec, 7b7614c157b34a33adc7552a2706e00c
I20260808 22:58:55.242416 21060 raft_consensus.cc:2454] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7614c157b34a33adc7552a2706e00c in term 1.
I20260808 22:58:55.242416 21198 raft_consensus.cc:2454] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7614c157b34a33adc7552a2706e00c in term 1.
I20260808 22:58:55.242533 21270 raft_consensus.cc:2726] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 22:58:55.242601 20849 leader_election.cc:302] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 585b15bd05374bfa8917433c78291e28, 7b7614c157b34a33adc7552a2706e00c; no voters:
I20260808 22:58:55.242715 21271 raft_consensus.cc:2781] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:55.242888 21271 raft_consensus.cc:686] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 LEADER]: Becoming Leader. State: Replica: 7b7614c157b34a33adc7552a2706e00c, State: Running, Role: LEADER
I20260808 22:58:55.243016 21271 consensus_queue.cc:237] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } }
I20260808 22:58:55.243976 20766 catalog_manager.cc:5495] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b7614c157b34a33adc7552a2706e00c (127.1.212.1). New cstate: current_term: 1 leader_uuid: "7b7614c157b34a33adc7552a2706e00c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b7614c157b34a33adc7552a2706e00c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 35033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 } health_report { overall_health: UNKNOWN } } }
W20260808 22:58:55.244856 20967 tablet.cc:2369] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:58:55.275123 21248 tablet.cc:2369] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:55.284917 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.282319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.1.212.1:35033)
0808 22:58:55.282404 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:55.282406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:55.282429 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:58:55.282470 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:55.282470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:55.282491 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:55.282547 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:55.282688 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.283176 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.283954 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:55.284173 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:55.284242 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:55.284481 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:55.284609 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:58:55.284688 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:55.284723 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 22:58:55.284749 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:58:55.286885 21060 raft_consensus.cc:1261] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 1 FOLLOWER]: Refusing update from remote peer 7b7614c157b34a33adc7552a2706e00c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:55.286886 21198 raft_consensus.cc:1261] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 1 FOLLOWER]: Refusing update from remote peer 7b7614c157b34a33adc7552a2706e00c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:55.287307 21271 consensus_queue.cc:1035] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [LEADER]: Connected to new peer: Peer: permanent_uuid: "585b15bd05374bfa8917433c78291e28" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:55.287484 21275 consensus_queue.cc:1035] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb7d8fbd43b4b358ff2d8df617ac1ec" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:55.290010 21278 mvcc.cc:205] Tried to move back new op lower bound from 7316397814932312064 to 7316397814756343808. Current Snapshot: MvccSnapshot[applied={T|T < 7316397814932312064}]
I20260808 22:58:55.291461 21279 mvcc.cc:205] Tried to move back new op lower bound from 7316397814932312064 to 7316397814756343808. Current Snapshot: MvccSnapshot[applied={T|T < 7316397814932312064 or (T in {7316397814932312064})}]
I20260808 22:58:55.292850 20766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:55.292948 20766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:55.293840 20766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 989453b49fb2498ea7f76e79ca6c674f: Sending DeleteTablet for 3 replicas of tablet 98ae17df523c4199af5b7a96b418ca45
I20260808 22:58:55.294276 21178 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae17df523c4199af5b7a96b418ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:55 UTC) from {username='slave'} at 127.0.0.1:36024
I20260808 22:58:55.294277 20896 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae17df523c4199af5b7a96b418ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:55 UTC) from {username='slave'} at 127.0.0.1:43048
I20260808 22:58:55.294466 21040 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ae17df523c4199af5b7a96b418ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:55 UTC) from {username='slave'} at 127.0.0.1:60586
I20260808 22:58:55.294548 21287 tablet_replica.cc:330] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: stopping tablet replica
I20260808 22:58:55.294665 21287 raft_consensus.cc:2229] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:55.294682 21288 tablet_replica.cc:330] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c: stopping tablet replica
I20260808 22:58:55.294801 21287 raft_consensus.cc:2258] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:55.295032 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20831
I20260808 22:58:55.295182 21288 raft_consensus.cc:2229] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:58:55.295390 21288 raft_consensus.cc:2258] T 98ae17df523c4199af5b7a96b418ca45 P 7b7614c157b34a33adc7552a2706e00c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:55.295485 21287 ts_tablet_manager.cc:1905] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:55.295758 21289 tablet_replica.cc:330] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: stopping tablet replica
I20260808 22:58:55.295894 21289 raft_consensus.cc:2229] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:55.296006 21289 raft_consensus.cc:2258] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:55.296633 21289 ts_tablet_manager.cc:1905] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:55.297684 21287 ts_tablet_manager.cc:1918] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:55.297778 21287 log.cc:1196] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/98ae17df523c4199af5b7a96b418ca45
I20260808 22:58:55.298115 21287 ts_tablet_manager.cc:1939] T 98ae17df523c4199af5b7a96b418ca45 P 0fb7d8fbd43b4b358ff2d8df617ac1ec: Deleting consensus metadata
I20260808 22:58:55.298638 20749 catalog_manager.cc:4841] TS 0fb7d8fbd43b4b358ff2d8df617ac1ec (127.1.212.3:43311): tablet 98ae17df523c4199af5b7a96b418ca45 (table test-table [id=158ae02eb310436aa760895ad25bef15]) successfully deleted
I20260808 22:58:55.299211 21289 ts_tablet_manager.cc:1918] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:55.299322 21289 log.cc:1196] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/98ae17df523c4199af5b7a96b418ca45
I20260808 22:58:55.300045 21289 ts_tablet_manager.cc:1939] T 98ae17df523c4199af5b7a96b418ca45 P 585b15bd05374bfa8917433c78291e28: Deleting consensus metadata
I20260808 22:58:55.300657 20751 catalog_manager.cc:4841] TS 585b15bd05374bfa8917433c78291e28 (127.1.212.2:44453): tablet 98ae17df523c4199af5b7a96b418ca45 (table test-table [id=158ae02eb310436aa760895ad25bef15]) successfully deleted
W20260808 22:58:55.306177 20750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:35033 (error 108)
I20260808 22:58:55.306331 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20969
I20260808 22:58:55.310065 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:55.309888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:35033 (local address 127.0.0.1:43068)
0808 22:58:55.309967 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:55.310023 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:35033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 22:58:55.310246 20750 catalog_manager.cc:4584] TS 7b7614c157b34a33adc7552a2706e00c (127.1.212.1:35033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98ae17df523c4199af5b7a96b418ca45: Network error: Client connection negotiation failed: client connection to 127.1.212.1:35033: connect: Connection refused (error 111)
I20260808 22:58:55.313969 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21112
I20260808 22:58:55.321085 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 20733
2026-08-08T22:58:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (913 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:58:55 dist-test-slave-1x4g krb5kdc[21293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:58:55 dist-test-slave-1x4g krb5kdc[21293](info): set up 2 sockets
Aug 08 22:58:55 dist-test-slave-1x4g krb5kdc[21293](info): commencing operation
krb5kdc: starting...
W20260808 22:58:56.424930 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:58:56 dist-test-slave-1x4g krb5kdc[21293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:58:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:58:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:58:56.564394 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35231
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:35231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:56.653761 21306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:56.654021 21306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:56.654048 21306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:56.655237 21306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:56.655282 21306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:56.655298 21306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:56.655313 21306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:56.655328 21306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:56.656896 21306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:35231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:56.657128 21306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:56.657325 21306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:58:56.662513 21306 server_base.cc:1034] running on GCE node
W20260808 22:58:56.662482 21313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:56.662487 21314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:56.662475 21316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:56.663067 21306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:56.663348 21306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:56.664538 21306 hybrid_clock.cc:648] HybridClock initialized: now 1786229936664537 us; error 40 us; skew 500 ppm
Aug 08 22:58:56 dist-test-slave-1x4g krb5kdc[21293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:56.672456 21306 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:58:56.672993 21306 webserver.cc:460] Webserver started at http://127.1.212.62:33079/ using document root <none> and password file <none>
I20260808 22:58:56.673242 21306 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:56.673313 21306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:56.673472 21306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:56.674624 21306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "8e6c06ad030748ea8abcefe8742e39f2"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "dd9a5b0306f381ed7397075cab711333"
server_key_iv: "12eb570b5c6ded647fc3a0a77422c473"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.675019 21306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "8e6c06ad030748ea8abcefe8742e39f2"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "dd9a5b0306f381ed7397075cab711333"
server_key_iv: "12eb570b5c6ded647fc3a0a77422c473"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.676474 21306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:58:56.677354 21323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:56.677556 21306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:56.677678 21306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "8e6c06ad030748ea8abcefe8742e39f2"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "dd9a5b0306f381ed7397075cab711333"
server_key_iv: "12eb570b5c6ded647fc3a0a77422c473"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.677758 21306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:56.691576 21306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:56.694134 21306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:56.694309 21306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:56.698652 21306 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:35231
I20260808 22:58:56.698688 21378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:35231 every 8 connection(s)
I20260808 22:58:56.699077 21306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:56.700201 21379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:56.701198 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21306
I20260808 22:58:56.701340 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:58:56.701539 1872 external_mini_cluster.cc:1428] Setting key f7b071292cd9abc759bd2d76815b3919
I20260808 22:58:56.703868 21379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: Bootstrap starting.
I20260808 22:58:56.704530 21379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:56.704917 21379 log.cc:824] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:56.705765 21379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: No bootstrap required, opened a new log
I20260808 22:58:56.708467 21379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } }
I20260808 22:58:56.708623 21379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:56.708647 21379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6c06ad030748ea8abcefe8742e39f2, State: Initialized, Role: FOLLOWER
I20260808 22:58:56.708815 21379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } }
I20260808 22:58:56.708909 21379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:56.708935 21379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:56.709007 21379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 22:58:56 dist-test-slave-1x4g krb5kdc[21293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:56.709643 21379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } }
I20260808 22:58:56.709738 21379 leader_election.cc:302] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e6c06ad030748ea8abcefe8742e39f2; no voters:
I20260808 22:58:56.709990 21379 leader_election.cc:288] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:58:56.710023 21384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:56.710199 21384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [term 1 LEADER]: Becoming Leader. State: Replica: 8e6c06ad030748ea8abcefe8742e39f2, State: Running, Role: LEADER
I20260808 22:58:56.710286 21379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:56.710287 21384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } }
I20260808 22:58:56.710629 21384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e6c06ad030748ea8abcefe8742e39f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } } }
I20260808 22:58:56.710666 21385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e6c06ad030748ea8abcefe8742e39f2. Latest consensus state: current_term: 1 leader_uuid: "8e6c06ad030748ea8abcefe8742e39f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6c06ad030748ea8abcefe8742e39f2" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35231 } } }
I20260808 22:58:56.710799 21384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:56.710850 21385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:56.711341 21389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:56.711908 21389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:56.713805 21389 catalog_manager.cc:1347] Generated new cluster ID: dfdebf667b8b428c8e47ddf491c4f418
I20260808 22:58:56.713862 21389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:56.714350 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:56.702133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.1.212.62:35231)
0808 22:58:56.702333 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:56.702345 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:56.702376 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:58:56.702812 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:56.702819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:56.702867 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:56.702992 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:56.703875 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.705039 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.706075 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.706525 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.706613 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:56.709611 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:56.709624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:56.709624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:56.709641 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:56.712259 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:56.712707 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:56.712711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:56.712712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:56.712841 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:56.713091 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:56.713092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:56.713092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:56.713423 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:56.713554 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:56.714007 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 22:58:56.714153 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260808 22:58:56.723093 21389 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:58:56.723704 21389 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:56.727592 21389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: Generated new TSK 0
I20260808 22:58:56.727795 21389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:58:56.760339 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:56.851996 21406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:56.852260 21406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:56.852306 21406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:56.853673 21406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:56.853751 21406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:56.853802 21406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:56.855382 21406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:56.855682 21406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:56.855937 21406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:56.861514 21411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:56.861498 21412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:56.861521 21414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:56.861828 21406 server_base.cc:1034] running on GCE node
I20260808 22:58:56.862102 21406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:56.862391 21406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:56.863575 21406 hybrid_clock.cc:648] HybridClock initialized: now 1786229936863558 us; error 28 us; skew 500 ppm
Aug 08 22:58:56 dist-test-slave-1x4g krb5kdc[21293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:56.866410 21406 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:58:56.867108 21406 webserver.cc:460] Webserver started at http://127.1.212.1:32793/ using document root <none> and password file <none>
I20260808 22:58:56.867439 21406 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:56.867509 21406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:56.867640 21406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:56.869201 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "77016c230be04d1091d0994564efb20c"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "b9e7a717d234d93b6641f3f276c6ff4c"
server_key_iv: "5e861f128525a1e720de255fbce42c05"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.869674 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "77016c230be04d1091d0994564efb20c"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "b9e7a717d234d93b6641f3f276c6ff4c"
server_key_iv: "5e861f128525a1e720de255fbce42c05"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.871304 21406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:58:56.872157 21422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:56.872416 21406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:56.872538 21406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "77016c230be04d1091d0994564efb20c"
format_stamp: "Formatted at 2026-08-08 22:58:56 on dist-test-slave-1x4g"
server_key: "b9e7a717d234d93b6641f3f276c6ff4c"
server_key_iv: "5e861f128525a1e720de255fbce42c05"
server_key_version: "encryptionkey@0"
I20260808 22:58:56.872640 21406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:56.913692 21406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:56.916241 21406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:56.916395 21406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:56.916710 21406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:56.917081 21406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:56.917119 21406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:56.917142 21406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:56.917189 21406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:56.923796 21406 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:37989
I20260808 22:58:56.923849 21545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:37989 every 8 connection(s)
I20260808 22:58:56.924201 21406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:56.927856 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21406
I20260808 22:58:56.928020 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:58:56.928234 1872 external_mini_cluster.cc:1428] Setting key 93cd8d3df81ef3114c6bd9d85cecd566
Aug 08 22:58:56 dist-test-slave-1x4g krb5kdc[21293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:56.935458 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:56.925050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:52051 (local address 127.1.212.62:35231)
0808 22:58:56.925103 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:56.925105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:56.925342 (+ 237us) server_negotiation.cc:407] Connection header received
0808 22:58:56.925874 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:56.925875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:56.925887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:56.925956 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:56.926971 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.927516 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.929032 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.929357 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.929439 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:56.931961 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:56.931964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:56.931964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:56.931974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:56.933541 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:56.934194 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:56.934195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:56.934195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:56.934258 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:56.934650 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:56.934651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:56.934651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:56.934792 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:56.934862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:56.935291 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 22:58:56.935384 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:58:56.935839 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:56.925013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35231 (local address 127.1.212.1:52051)
0808 22:58:56.925185 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:56.925238 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:56.925769 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:56.926014 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:56.926017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:56.926630 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:56.926842 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:56.926847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.927647 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.927648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:56.928937 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:56.928940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:56.929484 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:56.929485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:56.929620 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:56.930249 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:56.930258 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:56.931832 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:56.933645 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:56.933656 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:56.933659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:56.934070 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:56.934351 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:56.934353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:56.934353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:56.934548 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:56.934885 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:56.934895 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:56.935103 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 22:58:56.935251 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 22:58:56.935502 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260808 22:58:56.936494 21546 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35231
I20260808 22:58:56.936645 21546 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:56.936926 21546 heartbeater.cc:511] Master 127.1.212.62:35231 requested a full tablet report, sending...
I20260808 22:58:56.937549 21340 ts_manager.cc:194] Registered new tserver with Master: 77016c230be04d1091d0994564efb20c (127.1.212.1:37989)
I20260808 22:58:56.938203 21340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:52051
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:58:56.960405 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:57.047824 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:57.048079 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:57.048125 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:57.049489 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:57.049562 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:57.049611 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:57.051234 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:57.051563 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:57.051839 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:58:57.057442 21552 server_base.cc:1034] running on GCE node
W20260808 22:58:57.057466 21559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.057593 21558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.057442 21561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.057963 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:57.058234 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:57.059410 21552 hybrid_clock.cc:648] HybridClock initialized: now 1786229937059396 us; error 46 us; skew 500 ppm
Aug 08 22:58:57 dist-test-slave-1x4g krb5kdc[21293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229937, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:57.062292 21552 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:58:57.062815 21552 webserver.cc:460] Webserver started at http://127.1.212.2:36837/ using document root <none> and password file <none>
I20260808 22:58:57.063092 21552 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:57.063167 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:57.063329 21552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:57.064456 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "95865780e4e746f0a87f34c9d2e04daf"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "b3a67fc800cce8c815c848eb2788cae0"
server_key_iv: "ba76341a7b3899dc57b004711e4fc70a"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.064913 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "95865780e4e746f0a87f34c9d2e04daf"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "b3a67fc800cce8c815c848eb2788cae0"
server_key_iv: "ba76341a7b3899dc57b004711e4fc70a"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.066427 21552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:57.067225 21569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.067442 21552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:57.067560 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "95865780e4e746f0a87f34c9d2e04daf"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "b3a67fc800cce8c815c848eb2788cae0"
server_key_iv: "ba76341a7b3899dc57b004711e4fc70a"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.067655 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:57.077677 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:57.079768 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:57.079952 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:57.080238 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:57.080593 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:57.080650 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.080687 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:57.080734 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.087050 21552 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:44755
I20260808 22:58:57.087102 21687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:44755 every 8 connection(s)
I20260808 22:58:57.087466 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:58:57 dist-test-slave-1x4g krb5kdc[21293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229937, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:57.097010 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21552
I20260808 22:58:57.097139 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:58:57.097357 1872 external_mini_cluster.cc:1428] Setting key 998c55e22ae6c2e23fe262c10da2e0ca
I20260808 22:58:57.097988 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.088283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:53657 (local address 127.1.212.62:35231)
0808 22:58:57.088353 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.088356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.088568 (+ 212us) server_negotiation.cc:407] Connection header received
0808 22:58:57.089128 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.089129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.089140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.089205 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:57.090243 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.090830 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.091967 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.092169 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.092227 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.094648 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:57.094650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:57.094650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:57.094658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:57.096212 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.096704 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.096706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.096706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.096809 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.097106 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.097108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.097108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.097289 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:57.097372 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.097783 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.097895 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:58:57.098203 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.088248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35231 (local address 127.1.212.2:53657)
0808 22:58:57.088419 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.088473 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.089021 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.089287 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.089290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.089909 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:57.090134 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.090139 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.090980 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.090980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.091859 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.091862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.092257 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.092258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.092378 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.092987 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:57.092998 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:57.094483 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:57.096316 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:57.096328 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:57.096331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:57.096615 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:57.096893 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:57.096894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:57.096894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:57.097028 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:57.097373 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:57.097382 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:57.097636 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.097757 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.097943 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260808 22:58:57.098940 21688 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35231
I20260808 22:58:57.099079 21688 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:57.099280 21688 heartbeater.cc:511] Master 127.1.212.62:35231 requested a full tablet report, sending...
I20260808 22:58:57.099718 21340 ts_manager.cc:194] Registered new tserver with Master: 95865780e4e746f0a87f34c9d2e04daf (127.1.212.2:44755)
I20260808 22:58:57.100201 21340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:53657
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:58:57.128594 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:57.217028 21694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:57.217268 21694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:57.217295 21694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:57.218604 21694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:57.218662 21694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:57.218693 21694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:57.220369 21694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:57.220664 21694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:57.220969 21694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:57.226615 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.226615 21700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.226634 21699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.226898 21694 server_base.cc:1034] running on GCE node
I20260808 22:58:57.227145 21694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:57.227451 21694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:57.228655 21694 hybrid_clock.cc:648] HybridClock initialized: now 1786229937228634 us; error 37 us; skew 500 ppm
Aug 08 22:58:57 dist-test-slave-1x4g krb5kdc[21293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229937, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:57.231448 21694 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:58:57.231935 21694 webserver.cc:460] Webserver started at http://127.1.212.3:38311/ using document root <none> and password file <none>
I20260808 22:58:57.232183 21694 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:57.232255 21694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:57.232419 21694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:57.233680 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "64d19e040a8a42d7b25b31ba68a3c267"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "769f87a07e9154d20e1ca0a30b94498c"
server_key_iv: "873c9e8a72357d364237d7109d59f9ef"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.234077 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "64d19e040a8a42d7b25b31ba68a3c267"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "769f87a07e9154d20e1ca0a30b94498c"
server_key_iv: "873c9e8a72357d364237d7109d59f9ef"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.235622 21694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:58:57.236443 21709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.236634 21694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:57.236748 21694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "64d19e040a8a42d7b25b31ba68a3c267"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "769f87a07e9154d20e1ca0a30b94498c"
server_key_iv: "873c9e8a72357d364237d7109d59f9ef"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.236882 21694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:57.247872 21694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:57.250072 21694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:57.250257 21694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:57.250541 21694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:57.250891 21694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:57.250949 21694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.250991 21694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:57.251015 21694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.257507 21694 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:37495
I20260808 22:58:57.257546 21831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:37495 every 8 connection(s)
I20260808 22:58:57.257911 21694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:58:57 dist-test-slave-1x4g krb5kdc[21293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229937, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:57.265146 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21694
I20260808 22:58:57.265283 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:58:57.265506 1872 external_mini_cluster.cc:1428] Setting key 5cb5ad8a54bb7ef824368a8921be63a6
I20260808 22:58:57.268486 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.258725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:52143 (local address 127.1.212.62:35231)
0808 22:58:57.258796 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.258799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.259044 (+ 245us) server_negotiation.cc:407] Connection header received
0808 22:58:57.259579 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.259581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.259594 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.259661 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:57.260720 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.261306 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.262349 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.262544 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.262602 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.265137 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:57.265139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:57.265140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:57.265148 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:57.266888 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.267361 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.267363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.267364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.267437 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.267733 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.267734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.267735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.267874 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:57.267956 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.268277 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.268374 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:58:57.268719 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.258687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35231 (local address 127.1.212.3:52143)
0808 22:58:57.258892 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.258939 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.259471 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.259748 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.259752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.260397 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:57.260617 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.260623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.261404 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.261404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.262262 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.262265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.262627 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.262628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.262740 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.263412 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:57.263424 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:57.265020 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:57.266987 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:57.266998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:57.267001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:57.267273 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:57.267522 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:57.267523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:57.267523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:57.267659 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:57.267947 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:57.267957 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:57.268131 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.268253 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.268444 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
I20260808 22:58:57.269235 21832 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35231
I20260808 22:58:57.269342 21832 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:57.269567 21832 heartbeater.cc:511] Master 127.1.212.62:35231 requested a full tablet report, sending...
I20260808 22:58:57.270051 21340 ts_manager.cc:194] Registered new tserver with Master: 64d19e040a8a42d7b25b31ba68a3c267 (127.1.212.3:37495)
I20260808 22:58:57.270452 21340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:52143
I20260808 22:58:57.277601 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:58:57.286154 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.279113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.1.212.62:35231)
0808 22:58:57.279208 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.279211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.279307 (+ 96us) server_negotiation.cc:407] Connection header received
0808 22:58:57.279405 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.279406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.279417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.279501 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:57.280128 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.280703 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.281595 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.281799 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.281869 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.283468 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:57.283469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:57.283470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:57.283477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:57.284865 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.285310 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.285311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.285312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.285372 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:57.285594 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:57.285595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:57.285595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:57.285751 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:57.285839 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.285960 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.286017 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:58:57.288125 21340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:58:57.289923 21340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:58:57.297546 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.1.212.3:37495)
0808 22:58:57.292985 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.292991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.293220 (+ 229us) server_negotiation.cc:407] Connection header received
0808 22:58:57.293276 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.293279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.293296 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.293377 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:57.293750 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.294834 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.296209 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297049 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297136 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297211 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.297383 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.297442 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":137,"threads_started":1}
I20260808 22:58:57.297559 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:37495 (local address 127.0.0.1:38156)
0808 22:58:57.292984 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.293033 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.293119 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.293603 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.293604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.293606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:57.293667 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.293668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.294951 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.294952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.296102 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.296106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297146 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.297251 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297268 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.297354 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.297422 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":71,"thread_start_us":83,"threads_started":1}
I20260808 22:58:57.297961 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.1.212.2:44755)
0808 22:58:57.293033 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.293038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.293238 (+ 200us) server_negotiation.cc:407] Connection header received
0808 22:58:57.293281 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.293283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.293297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.293376 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:57.293587 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.294828 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.296380 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297137 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297202 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297251 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.297603 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.297656 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
I20260808 22:58:57.298026 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37989 (local address 127.0.0.1:51458)
0808 22:58:57.293493 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.293513 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.293566 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.293808 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.293809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.293811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:57.293875 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.293876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.294781 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.294782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.296275 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.296278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297743 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.297845 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297850 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.297905 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.297952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":60,"thread_start_us":75,"threads_started":1}
I20260808 22:58:57.298271 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.1.212.1:37989)
0808 22:58:57.293144 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.293148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.293637 (+ 489us) server_negotiation.cc:407] Connection header received
0808 22:58:57.293689 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.293692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.293708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.293784 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:57.293955 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.294667 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.296866 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297569 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297642 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297704 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.298122 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.298178 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260808 22:58:57.298614 21760 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 (DEFAULT_TABLE table=test-table [id=889c1659062d453c8b6bd8c0d10bb775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:57.299052 21760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba001adf1324c50bfc5e5f541dea3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:57.298957 21476 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 (DEFAULT_TABLE table=test-table [id=889c1659062d453c8b6bd8c0d10bb775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:57.299299 21476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba001adf1324c50bfc5e5f541dea3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:57.300982 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.292568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44755 (local address 127.0.0.1:38262)
0808 22:58:57.293080 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.293089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.293136 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.293393 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.293397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.293400 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:57.293496 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.293498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.295042 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.295043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.296275 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.296278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.297294 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.297295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.297380 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.297384 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.297439 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.297482 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":7,"thread_start_us":78,"threads_started":1}
I20260808 22:58:57.301945 21620 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 (DEFAULT_TABLE table=test-table [id=889c1659062d453c8b6bd8c0d10bb775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:57.302318 21620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba001adf1324c50bfc5e5f541dea3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:57.303007 21852 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Bootstrap starting.
I20260808 22:58:57.303254 21851 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Bootstrap starting.
I20260808 22:58:57.304072 21851 tablet_bootstrap.cc:654] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:57.304716 21851 log.cc:824] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:57.305642 21852 tablet_bootstrap.cc:654] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:57.305706 21851 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: No bootstrap required, opened a new log
I20260808 22:58:57.305783 21851 ts_tablet_manager.cc:1397] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:58:57.306082 21854 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Bootstrap starting.
I20260808 22:58:57.306317 21852 log.cc:824] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:57.306844 21854 tablet_bootstrap.cc:654] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:57.307293 21852 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: No bootstrap required, opened a new log
I20260808 22:58:57.307380 21852 ts_tablet_manager.cc:1397] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:58:57.307502 21854 log.cc:824] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:57.307928 21851 raft_consensus.cc:348] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.308156 21851 raft_consensus.cc:374] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:57.308207 21851 raft_consensus.cc:729] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77016c230be04d1091d0994564efb20c, State: Initialized, Role: FOLLOWER
I20260808 22:58:57.308336 21851 consensus_queue.cc:260] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.308372 21854 tablet_bootstrap.cc:492] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: No bootstrap required, opened a new log
I20260808 22:58:57.308442 21854 ts_tablet_manager.cc:1397] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:58:57.308662 21851 ts_tablet_manager.cc:1428] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:57.308838 21546 heartbeater.cc:503] Master 127.1.212.62:35231 was elected leader, sending a full tablet report...
I20260808 22:58:57.309636 21852 raft_consensus.cc:348] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.309831 21852 raft_consensus.cc:374] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:57.309880 21852 raft_consensus.cc:729] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d19e040a8a42d7b25b31ba68a3c267, State: Initialized, Role: FOLLOWER
I20260808 22:58:57.309994 21852 consensus_queue.cc:260] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.310353 21852 ts_tablet_manager.cc:1428] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:58:57.310503 21832 heartbeater.cc:503] Master 127.1.212.62:35231 was elected leader, sending a full tablet report...
I20260808 22:58:57.311074 21854 raft_consensus.cc:348] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.311288 21854 raft_consensus.cc:374] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:57.311331 21854 raft_consensus.cc:729] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95865780e4e746f0a87f34c9d2e04daf, State: Initialized, Role: FOLLOWER
I20260808 22:58:57.311425 21854 consensus_queue.cc:260] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.311726 21854 ts_tablet_manager.cc:1428] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:58:57.311733 21688 heartbeater.cc:503] Master 127.1.212.62:35231 was elected leader, sending a full tablet report...
W20260808 22:58:57.338171 21689 tablet.cc:2369] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:58:57.425050 21547 tablet.cc:2369] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:57.467885 21858 raft_consensus.cc:482] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:57.468062 21858 raft_consensus.cc:504] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.468519 21858 leader_election.cc:288] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95865780e4e746f0a87f34c9d2e04daf (127.1.212.2:44755), 77016c230be04d1091d0994564efb20c (127.1.212.1:37989)
I20260808 22:58:57.471726 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.468724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:44755 (local address 127.1.212.3:37903)
0808 22:58:57.468808 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.468822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.468872 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.469052 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.469054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.469056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:57.469149 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.469151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.469856 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.469858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.470718 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.470721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.471432 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.471433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.471529 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.471534 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.471602 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.471653 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":50}
I20260808 22:58:57.471751 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.468835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:37903 (local address 127.1.212.2:44755)
0808 22:58:57.468885 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.468887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.468899 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:58:57.468946 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.468947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.468960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.469020 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:57.469232 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.469769 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.470813 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.471349 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.471420 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.471432 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.471629 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.471671 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:58:57.471932 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.468780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37989 (local address 127.1.212.3:57735)
0808 22:58:57.468980 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.468989 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.469039 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.469205 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.469206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.469208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:57.469288 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.469289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.469958 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.469959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.470936 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.470939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.471640 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.471641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.471712 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.471717 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.471778 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.471823 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":134,"threads_started":1}
I20260808 22:58:57.471942 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.468887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:57735 (local address 127.1.212.1:37989)
0808 22:58:57.468945 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.468947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.469051 (+ 104us) server_negotiation.cc:407] Connection header received
0808 22:58:57.469088 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.469089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.469102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.469176 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:57.469364 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.469850 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.471052 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.471535 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.471611 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.471621 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.471804 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.471855 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:58:57.472148 21640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba001adf1324c50bfc5e5f541dea3e6" candidate_uuid: "64d19e040a8a42d7b25b31ba68a3c267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95865780e4e746f0a87f34c9d2e04daf" is_pre_election: true
I20260808 22:58:57.472339 21640 raft_consensus.cc:2454] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d19e040a8a42d7b25b31ba68a3c267 in term 0.
I20260808 22:58:57.472361 21496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba001adf1324c50bfc5e5f541dea3e6" candidate_uuid: "64d19e040a8a42d7b25b31ba68a3c267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77016c230be04d1091d0994564efb20c" is_pre_election: true
I20260808 22:58:57.472558 21496 raft_consensus.cc:2454] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d19e040a8a42d7b25b31ba68a3c267 in term 0.
I20260808 22:58:57.472627 21714 leader_election.cc:302] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d19e040a8a42d7b25b31ba68a3c267, 95865780e4e746f0a87f34c9d2e04daf; no voters:
I20260808 22:58:57.472818 21858 raft_consensus.cc:2781] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:58:57.472885 21858 raft_consensus.cc:482] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:58:57.472904 21858 raft_consensus.cc:3037] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:57.473719 21858 raft_consensus.cc:504] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.473887 21858 leader_election.cc:288] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [CANDIDATE]: Term 1 election: Requested vote from peers 95865780e4e746f0a87f34c9d2e04daf (127.1.212.2:44755), 77016c230be04d1091d0994564efb20c (127.1.212.1:37989)
I20260808 22:58:57.474095 21496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba001adf1324c50bfc5e5f541dea3e6" candidate_uuid: "64d19e040a8a42d7b25b31ba68a3c267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77016c230be04d1091d0994564efb20c"
I20260808 22:58:57.474162 21640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba001adf1324c50bfc5e5f541dea3e6" candidate_uuid: "64d19e040a8a42d7b25b31ba68a3c267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95865780e4e746f0a87f34c9d2e04daf"
I20260808 22:58:57.474200 21496 raft_consensus.cc:3037] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:57.474282 21640 raft_consensus.cc:3037] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:57.475095 21496 raft_consensus.cc:2454] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d19e040a8a42d7b25b31ba68a3c267 in term 1.
I20260808 22:58:57.475123 21640 raft_consensus.cc:2454] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d19e040a8a42d7b25b31ba68a3c267 in term 1.
I20260808 22:58:57.475287 21714 leader_election.cc:302] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d19e040a8a42d7b25b31ba68a3c267, 95865780e4e746f0a87f34c9d2e04daf; no voters:
I20260808 22:58:57.475409 21858 raft_consensus.cc:2781] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:57.475641 21858 raft_consensus.cc:686] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 1 LEADER]: Becoming Leader. State: Replica: 64d19e040a8a42d7b25b31ba68a3c267, State: Running, Role: LEADER
I20260808 22:58:57.475759 21858 consensus_queue.cc:237] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } }
I20260808 22:58:57.476765 21339 catalog_manager.cc:5495] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64d19e040a8a42d7b25b31ba68a3c267 (127.1.212.3). New cstate: current_term: 1 leader_uuid: "64d19e040a8a42d7b25b31ba68a3c267" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64d19e040a8a42d7b25b31ba68a3c267" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 } health_report { overall_health: UNKNOWN } } }
W20260808 22:58:57.508555 21834 tablet.cc:2369] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:58:57.526162 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.523632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.1.212.3:37495)
0808 22:58:57.523700 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.523702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.523715 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:58:57.523778 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.523779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.523802 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.523899 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:57.524094 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.524571 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.525420 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.525614 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.525671 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.525816 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:57.525911 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:58:57.525982 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.526054 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.526063 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:58:57.528214 21496 raft_consensus.cc:1261] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 1 FOLLOWER]: Refusing update from remote peer 64d19e040a8a42d7b25b31ba68a3c267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:57.528226 21640 raft_consensus.cc:1261] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 1 FOLLOWER]: Refusing update from remote peer 64d19e040a8a42d7b25b31ba68a3c267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:57.528656 21861 consensus_queue.cc:1035] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95865780e4e746f0a87f34c9d2e04daf" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:57.528831 21858 consensus_queue.cc:1035] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77016c230be04d1091d0994564efb20c" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 37989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:57.531571 21865 mvcc.cc:205] Tried to move back new op lower bound from 7316397824112619520 to 7316397823901585408. Current Snapshot: MvccSnapshot[applied={T|T < 7316397824112619520}]
I20260808 22:58:57.534384 21339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:57.534516 21339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:57.535347 21339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e6c06ad030748ea8abcefe8742e39f2: Sending DeleteTablet for 3 replicas of tablet 8ba001adf1324c50bfc5e5f541dea3e6
I20260808 22:58:57.535707 21620 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:38262
I20260808 22:58:57.535769 21476 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:51458
I20260808 22:58:57.535799 21760 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ba001adf1324c50bfc5e5f541dea3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:57 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:38156
I20260808 22:58:57.536037 21876 tablet_replica.cc:330] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c: stopping tablet replica
I20260808 22:58:57.536160 21876 raft_consensus.cc:2229] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:57.536263 21874 tablet_replica.cc:330] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: stopping tablet replica
I20260808 22:58:57.536298 21876 raft_consensus.cc:2258] T 8ba001adf1324c50bfc5e5f541dea3e6 P 77016c230be04d1091d0994564efb20c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:57.536398 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21406
I20260808 22:58:57.536461 21875 tablet_replica.cc:330] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: stopping tablet replica
I20260808 22:58:57.536558 21875 raft_consensus.cc:2229] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:58:57.536670 21875 raft_consensus.cc:2258] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:57.543179 21874 raft_consensus.cc:2229] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:57.543447 21874 raft_consensus.cc:2258] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:57.543634 21875 ts_tablet_manager.cc:1905] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:57.544126 21874 ts_tablet_manager.cc:1905] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:57.546156 21875 ts_tablet_manager.cc:1918] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:57.546156 21874 ts_tablet_manager.cc:1918] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:57.546249 21875 log.cc:1196] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/8ba001adf1324c50bfc5e5f541dea3e6
I20260808 22:58:57.546396 21874 log.cc:1196] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/8ba001adf1324c50bfc5e5f541dea3e6
I20260808 22:58:57.546577 21875 ts_tablet_manager.cc:1939] T 8ba001adf1324c50bfc5e5f541dea3e6 P 64d19e040a8a42d7b25b31ba68a3c267: Deleting consensus metadata
I20260808 22:58:57.546847 21874 ts_tablet_manager.cc:1939] T 8ba001adf1324c50bfc5e5f541dea3e6 P 95865780e4e746f0a87f34c9d2e04daf: Deleting consensus metadata
I20260808 22:58:57.547070 21325 catalog_manager.cc:4841] TS 64d19e040a8a42d7b25b31ba68a3c267 (127.1.212.3:37495): tablet 8ba001adf1324c50bfc5e5f541dea3e6 (table test-table [id=889c1659062d453c8b6bd8c0d10bb775]) successfully deleted
I20260808 22:58:57.547473 21327 catalog_manager.cc:4841] TS 95865780e4e746f0a87f34c9d2e04daf (127.1.212.2:44755): tablet 8ba001adf1324c50bfc5e5f541dea3e6 (table test-table [id=889c1659062d453c8b6bd8c0d10bb775]) successfully deleted
W20260808 22:58:57.550298 21325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:37989 (error 108)
I20260808 22:58:57.550942 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.550773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:37989 (local address 127.0.0.1:51470)
0808 22:58:57.550838 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.550880 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 22:58:57.551013 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21552
W20260808 22:58:57.557417 21325 catalog_manager.cc:4584] TS 77016c230be04d1091d0994564efb20c (127.1.212.1:37989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ba001adf1324c50bfc5e5f541dea3e6: Network error: Client connection negotiation failed: client connection to 127.1.212.1:37989: connect: Connection refused (error 111)
I20260808 22:58:57.564092 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21694
I20260808 22:58:57.572479 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21306
2026-08-08T22:58:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2252 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T22:58:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:58:57Z Disabled control of system clock
I20260808 22:58:57.604496 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35173
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:57.690828 21881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:57.691082 21881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:57.691104 21881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:57.692287 21881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:57.692328 21881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:57.692341 21881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:57.692354 21881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:57.692368 21881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:57.693740 21881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:35173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:35173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:57.693979 21881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:57.694239 21881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:57.699689 21887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.699689 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.699751 21881 server_base.cc:1034] running on GCE node
W20260808 22:58:57.699689 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.700281 21881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:57.700621 21881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:57.701795 21881 hybrid_clock.cc:648] HybridClock initialized: now 1786229937701787 us; error 58 us; skew 500 ppm
I20260808 22:58:57.703261 21881 webserver.cc:460] Webserver started at http://127.1.212.62:40047/ using document root <none> and password file <none>
I20260808 22:58:57.703534 21881 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:57.703610 21881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:57.703770 21881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:57.705029 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "37d1b79c9eab4ece8049cd84660e5a3a"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "df80e50102a058e258f6aa5f1b091717"
server_key_iv: "d396851e462e7b250f02c18603487663"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.705430 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "37d1b79c9eab4ece8049cd84660e5a3a"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "df80e50102a058e258f6aa5f1b091717"
server_key_iv: "d396851e462e7b250f02c18603487663"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.706872 21881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:58:57.707765 21897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.707993 21881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:57.708117 21881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "37d1b79c9eab4ece8049cd84660e5a3a"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "df80e50102a058e258f6aa5f1b091717"
server_key_iv: "d396851e462e7b250f02c18603487663"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.708199 21881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:57.721491 21881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:57.721884 21881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:57.722065 21881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:57.726868 21881 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:35173
I20260808 22:58:57.726907 21953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:35173 every 8 connection(s)
I20260808 22:58:57.727334 21881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:57.728408 21954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:57.729822 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21881
I20260808 22:58:57.729939 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:58:57.730156 1872 external_mini_cluster.cc:1428] Setting key f5aacf2b288a72c872dc807531233d3d
I20260808 22:58:57.731493 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: Bootstrap starting.
I20260808 22:58:57.732148 21954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:57.732460 21954 log.cc:824] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:57.733282 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: No bootstrap required, opened a new log
I20260808 22:58:57.735124 21954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } }
I20260808 22:58:57.735275 21954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:57.735298 21954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37d1b79c9eab4ece8049cd84660e5a3a, State: Initialized, Role: FOLLOWER
I20260808 22:58:57.735446 21954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } }
I20260808 22:58:57.735546 21954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:57.735603 21954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:57.735677 21954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:57.736122 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.730773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.1.212.62:35173)
0808 22:58:57.730986 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.730994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.731030 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:58:57.731455 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.731462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.731526 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.731737 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:57.731977 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.733153 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.734001 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.734458 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.734603 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.735391 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:57.735395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:57.735395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:57.735643 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:57.735689 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:57.735704 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:57.735762 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.735841 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.735928 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260808 22:58:57.736529 21954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } }
I20260808 22:58:57.736670 21954 leader_election.cc:302] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37d1b79c9eab4ece8049cd84660e5a3a; no voters:
I20260808 22:58:57.736901 21954 leader_election.cc:288] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:58:57.736955 21959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:57.737118 21959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [term 1 LEADER]: Becoming Leader. State: Replica: 37d1b79c9eab4ece8049cd84660e5a3a, State: Running, Role: LEADER
I20260808 22:58:57.737243 21959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } }
I20260808 22:58:57.737329 21954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:57.737635 21959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37d1b79c9eab4ece8049cd84660e5a3a. Latest consensus state: current_term: 1 leader_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } } }
I20260808 22:58:57.737644 21960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37d1b79c9eab4ece8049cd84660e5a3a" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 35173 } } }
I20260808 22:58:57.737730 21959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:57.737762 21960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a [sys.catalog]: This master's current role is: LEADER
W20260808 22:58:57.739168 21975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:58:57.739241 21975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:58:57.739319 21964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:57.739524 21964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:57.741225 21964 catalog_manager.cc:1347] Generated new cluster ID: bb615fe9597b4c8d90309420f091d318
I20260808 22:58:57.741291 21964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:58:57.747552 21964 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:58:57.748363 21964 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:57.753944 21964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: Generated new TSK 0
I20260808 22:58:57.754191 21964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:58:57.757869 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--builtin_ntp_servers=127.1.212.20:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:57.847613 21978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:57.847856 21978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:57.847896 21978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:57.849366 21978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:57.849421 21978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:57.849453 21978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:57.851039 21978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:57.851382 21978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:57.851758 21978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:57.857331 21984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.857394 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.857311 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.858049 21978 server_base.cc:1034] running on GCE node
I20260808 22:58:57.858291 21978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:57.858572 21978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:57.859759 21978 hybrid_clock.cc:648] HybridClock initialized: now 1786229937859733 us; error 40 us; skew 500 ppm
I20260808 22:58:57.861359 21978 webserver.cc:460] Webserver started at http://127.1.212.1:34791/ using document root <none> and password file <none>
I20260808 22:58:57.861655 21978 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:57.861737 21978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:57.861898 21978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:57.863111 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "c8e138e59b47dc78e4c15d7961343076"
server_key_iv: "115d065cd4017d72409fbe7a21b00d5e"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.863508 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "c8e138e59b47dc78e4c15d7961343076"
server_key_iv: "115d065cd4017d72409fbe7a21b00d5e"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.865157 21978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 22:58:57.866127 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.866344 21978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:57.866465 21978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "c8e138e59b47dc78e4c15d7961343076"
server_key_iv: "115d065cd4017d72409fbe7a21b00d5e"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.866559 21978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:57.877895 21978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:57.878297 21978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:57.878476 21978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:57.878748 21978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:57.879124 21978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:57.879184 21978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.879228 21978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:57.879256 21978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:57.885738 21978 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:36017
I20260808 22:58:57.885780 22111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:36017 every 8 connection(s)
I20260808 22:58:57.886170 21978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:57.892202 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.887025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:54005 (local address 127.1.212.62:35173)
0808 22:58:57.887147 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:57.887149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:57.887463 (+ 314us) server_negotiation.cc:407] Connection header received
0808 22:58:57.888056 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:57.888058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:57.888072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:57.888130 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:57.888956 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.889475 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.890561 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.890768 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.890851 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.891843 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:57.891845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:57.891845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:57.891852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:57.891877 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:57.891882 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:57.891961 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:57.892076 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 22:58:57.892111 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 22:58:57.892478 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:57.887113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35173 (local address 127.1.212.1:54005)
0808 22:58:57.887323 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:57.887362 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:57.887924 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:57.888221 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:57.888226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:57.888243 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:57.888849 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.888858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.889587 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.889588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.890441 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:57.890444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:57.890864 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:57.890865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:57.891029 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:57.891668 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:57.891669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:57.891703 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:57.891733 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:57.891966 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:57.891974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:57.891977 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:58:57.892066 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:58:57.892195 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260808 22:58:57.893144 22112 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35173
I20260808 22:58:57.893265 22112 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:57.893482 22112 heartbeater.cc:511] Master 127.1.212.62:35173 requested a full tablet report, sending...
I20260808 22:58:57.894131 21916 ts_manager.cc:194] Registered new tserver with Master: b1a0b8eebfad46bcb0f8cdd5a64e37f1 (127.1.212.1:36017)
I20260808 22:58:57.894656 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 21978
I20260808 22:58:57.894764 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:58:57.894959 1872 external_mini_cluster.cc:1428] Setting key e2cb12cfb16df652ceeb77534b1e1a5c
I20260808 22:58:57.895184 21916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.1:54005
I20260808 22:58:57.896189 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--builtin_ntp_servers=127.1.212.20:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:58:57.982729 22115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:57.983016 22115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:57.983069 22115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:57.984390 22115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:57.984472 22115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:57.984522 22115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:57.986107 22115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:57.986423 22115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:57.986901 22115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:57.992738 22122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.992738 22124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:57.992758 22121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:57.992911 22115 server_base.cc:1034] running on GCE node
I20260808 22:58:57.993270 22115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:57.993549 22115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:57.994727 22115 hybrid_clock.cc:648] HybridClock initialized: now 1786229937994709 us; error 51 us; skew 500 ppm
I20260808 22:58:57.996097 22115 webserver.cc:460] Webserver started at http://127.1.212.2:37805/ using document root <none> and password file <none>
I20260808 22:58:57.996376 22115 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:57.996448 22115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:57.996619 22115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:57.997831 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "61ce5db2a4c643b3af508f07bf621246"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "a501858df82a19bfa9131f136e66e2c0"
server_key_iv: "85ba3f7eff66c92058e2deb6a7b3a484"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.998227 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "61ce5db2a4c643b3af508f07bf621246"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "a501858df82a19bfa9131f136e66e2c0"
server_key_iv: "85ba3f7eff66c92058e2deb6a7b3a484"
server_key_version: "encryptionkey@0"
I20260808 22:58:57.999782 22115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:58:58.000665 22132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.000923 22115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:58.001035 22115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "61ce5db2a4c643b3af508f07bf621246"
format_stamp: "Formatted at 2026-08-08 22:58:57 on dist-test-slave-1x4g"
server_key: "a501858df82a19bfa9131f136e66e2c0"
server_key_iv: "85ba3f7eff66c92058e2deb6a7b3a484"
server_key_version: "encryptionkey@0"
I20260808 22:58:58.001127 22115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:58.013314 22115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:58.013656 22115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:58.013809 22115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:58.014062 22115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:58.014424 22115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:58.014480 22115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.014529 22115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:58.014566 22115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.020592 22115 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:40919
I20260808 22:58:58.020639 22250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:40919 every 8 connection(s)
I20260808 22:58:58.021054 22115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:58:58.022493 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22115
I20260808 22:58:58.022624 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:58:58.022820 1872 external_mini_cluster.cc:1428] Setting key 8f2bafa7d200339583393539444cc8ea
I20260808 22:58:58.023870 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--builtin_ntp_servers=127.1.212.20:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:58:58.026891 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.021794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:41053 (local address 127.1.212.62:35173)
0808 22:58:58.021839 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.021841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.022101 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:58:58.022632 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.022633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.022648 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.022708 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:58.023571 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.024190 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.025295 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.025616 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.025691 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.026575 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:58.026577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:58.026577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:58.026582 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:58.026601 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:58.026606 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:58.026667 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.026746 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.026776 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:58:58.027030 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.021757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35173 (local address 127.1.212.2:41053)
0808 22:58:58.021959 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.022003 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.022513 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.022788 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.022791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.022800 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:58.023467 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.023475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.024300 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.024303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.025201 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.025204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.025711 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.025712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.025839 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.026424 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:58.026424 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:58.026454 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:58.026479 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:58.026673 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:58.026677 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:58.026680 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.026735 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.026822 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260808 22:58:58.027634 22251 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35173
I20260808 22:58:58.027729 22251 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:58.027935 22251 heartbeater.cc:511] Master 127.1.212.62:35173 requested a full tablet report, sending...
I20260808 22:58:58.028378 21915 ts_manager.cc:194] Registered new tserver with Master: 61ce5db2a4c643b3af508f07bf621246 (127.1.212.2:40919)
I20260808 22:58:58.028972 21915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.2:41053
W20260808 22:58:58.110193 22254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:58.110478 22254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:58.110514 22254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:58.111792 22254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:58.111837 22254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:58.111864 22254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:58:58.113306 22254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:58.113526 22254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:58.113787 22254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:58.119128 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:58.119118 22264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:58.119236 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:58.119369 22254 server_base.cc:1034] running on GCE node
I20260808 22:58:58.119604 22254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:58.119910 22254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:58.121068 22254 hybrid_clock.cc:648] HybridClock initialized: now 1786229938121050 us; error 38 us; skew 500 ppm
I20260808 22:58:58.122521 22254 webserver.cc:460] Webserver started at http://127.1.212.3:43557/ using document root <none> and password file <none>
I20260808 22:58:58.122793 22254 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:58.122846 22254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:58.123030 22254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:58.124169 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "67dbbaca50c34949b79dab2037cc3cb4"
format_stamp: "Formatted at 2026-08-08 22:58:58 on dist-test-slave-1x4g"
server_key: "786b7b1a4e471fee3feab3c720b019b0"
server_key_iv: "49f13046ad03a7029d88825e2ab35cbb"
server_key_version: "encryptionkey@0"
I20260808 22:58:58.124557 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "67dbbaca50c34949b79dab2037cc3cb4"
format_stamp: "Formatted at 2026-08-08 22:58:58 on dist-test-slave-1x4g"
server_key: "786b7b1a4e471fee3feab3c720b019b0"
server_key_iv: "49f13046ad03a7029d88825e2ab35cbb"
server_key_version: "encryptionkey@0"
I20260808 22:58:58.126019 22254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:58:58.126845 22270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.127033 22254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:58.127133 22254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "67dbbaca50c34949b79dab2037cc3cb4"
format_stamp: "Formatted at 2026-08-08 22:58:58 on dist-test-slave-1x4g"
server_key: "786b7b1a4e471fee3feab3c720b019b0"
server_key_iv: "49f13046ad03a7029d88825e2ab35cbb"
server_key_version: "encryptionkey@0"
I20260808 22:58:58.127215 22254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:58.144279 22254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:58.144658 22254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:58.144876 22254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:58.145159 22254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:58.145545 22254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:58.145599 22254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.145637 22254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:58.145664 22254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:58.151808 22254 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:43755
I20260808 22:58:58.151861 22389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:43755 every 8 connection(s)
I20260808 22:58:58.152261 22254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:58:58.158039 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.153133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:57627 (local address 127.1.212.62:35173)
0808 22:58:58.153198 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.153200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.153453 (+ 253us) server_negotiation.cc:407] Connection header received
0808 22:58:58.154057 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.154058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.154075 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.154132 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:58.154868 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.155380 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.156404 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.156634 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.156734 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.157700 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:58.157702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:58.157702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:58.157708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:58.157724 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:58.157729 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:58.157804 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.157910 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.157944 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:58:58.158269 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.153099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:35173 (local address 127.1.212.3:57627)
0808 22:58:58.153326 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.153363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.153907 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.154212 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.154216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.154225 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:58.154763 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.154768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.155490 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.155491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.156303 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.156305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.156750 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.156751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.156914 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.157531 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:58:58.157532 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:58.157562 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:58:58.157589 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:58.157809 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:58.157815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:58.157819 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.157910 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.158025 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260808 22:58:58.158793 22390 heartbeater.cc:347] Connected to a master server at 127.1.212.62:35173
I20260808 22:58:58.158917 22390 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:58.159128 22390 heartbeater.cc:511] Master 127.1.212.62:35173 requested a full tablet report, sending...
I20260808 22:58:58.159564 21915 ts_manager.cc:194] Registered new tserver with Master: 67dbbaca50c34949b79dab2037cc3cb4 (127.1.212.3:43755)
I20260808 22:58:58.159953 21915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.3:57627
I20260808 22:58:58.160854 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22254
I20260808 22:58:58.160969 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:58:58.161185 1872 external_mini_cluster.cc:1428] Setting key 52415130646d35c415c099ed0a9a339a
I20260808 22:58:58.162144 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:58:58.166661 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.163594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49058 (local address 127.1.212.62:35173)
0808 22:58:58.163647 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.163649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.163711 (+ 62us) server_negotiation.cc:407] Connection header received
0808 22:58:58.163785 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.163786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.163795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.163837 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:58.164003 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.164497 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.165337 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.165522 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.165581 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.166408 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:58.166409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:58.166409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:58:58.166413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:58.166426 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:58:58.166430 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:58.166502 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.166593 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.166604 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:58:58.168426 21915 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:58:58.170181 21915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:58:58.177297 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.172738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43755 (local address 127.0.0.1:39328)
0808 22:58:58.172946 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.172991 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.173062 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.173812 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.173813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.173816 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:58.173877 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.173879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.174789 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.174790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.176100 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.176103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.176946 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.176948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.177043 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177061 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.177134 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.177197 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260808 22:58:58.177400 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.172738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36017 (local address 127.0.0.1:45228)
0808 22:58:58.172945 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.172991 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.173062 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.173662 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.173666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.173669 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:58.173752 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.173754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.175154 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.175155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.176210 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.176212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.177136 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.177137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.177190 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177193 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.177246 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.177277 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":2,"thread_start_us":130,"threads_started":1}
I20260808 22:58:58.177590 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.172750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:40919 (local address 127.0.0.1:59096)
0808 22:58:58.173425 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.173436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.173508 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.173748 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.173748 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.173750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:58.173784 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.173785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.175166 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.175167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.176097 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.176099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.177208 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.177209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.177275 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177279 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.177333 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.177376 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":158,"threads_started":1}
I20260808 22:58:58.177608 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.173077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.1.212.2:40919)
0808 22:58:58.173248 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.173252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.173517 (+ 265us) server_negotiation.cc:407] Connection header received
0808 22:58:58.173585 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.173588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.173603 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.173679 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:58.173872 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.175057 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.176191 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.176980 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.177060 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177166 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.177456 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.177512 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260808 22:58:58.178017 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.172941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.1.212.3:43755)
0808 22:58:58.173648 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.173653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.173663 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:58:58.173707 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.173709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.173726 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.173804 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:58.173962 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.174698 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.176207 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.176834 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.176933 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177045 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.177566 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.177621 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":85,"threads_started":1}
I20260808 22:58:58.178131 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.173247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45228 (local address 127.1.212.1:36017)
0808 22:58:58.173436 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.173440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.173450 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:58:58.173517 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.173519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.173539 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.173836 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:58.173903 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.175057 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.176285 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.177059 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.177120 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.177199 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.177258 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.177304 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260808 22:58:58.178445 22181 tablet_service.cc:1467] Processing CreateTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 (DEFAULT_TABLE table=test-table [id=916baca67f2a40139c00abaf72d196df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:58.178879 22181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86cf5aee09c64aa3bec96e821a836d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:58.178861 22044 tablet_service.cc:1467] Processing CreateTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 (DEFAULT_TABLE table=test-table [id=916baca67f2a40139c00abaf72d196df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:58.179234 22044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86cf5aee09c64aa3bec96e821a836d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:58.181577 22321 tablet_service.cc:1467] Processing CreateTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 (DEFAULT_TABLE table=test-table [id=916baca67f2a40139c00abaf72d196df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:58:58.181906 22405 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Bootstrap starting.
I20260808 22:58:58.181998 22321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86cf5aee09c64aa3bec96e821a836d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:58.182493 22406 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: Bootstrap starting.
I20260808 22:58:58.182642 22405 tablet_bootstrap.cc:654] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:58.182888 22405 log.cc:824] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:58.183287 22406 tablet_bootstrap.cc:654] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:58.183588 22405 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: No bootstrap required, opened a new log
I20260808 22:58:58.183635 22405 ts_tablet_manager.cc:1397] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:58:58.183645 22406 log.cc:824] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:58.184533 22406 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: No bootstrap required, opened a new log
I20260808 22:58:58.184604 22406 ts_tablet_manager.cc:1397] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:58:58.186085 22405 raft_consensus.cc:348] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.186276 22409 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Bootstrap starting.
I20260808 22:58:58.186317 22405 raft_consensus.cc:374] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:58.186478 22405 raft_consensus.cc:729] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5db2a4c643b3af508f07bf621246, State: Initialized, Role: FOLLOWER
I20260808 22:58:58.186650 22405 consensus_queue.cc:260] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.186964 22405 ts_tablet_manager.cc:1428] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 22:58:58.187094 22409 tablet_bootstrap.cc:654] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:58.187161 22251 heartbeater.cc:503] Master 127.1.212.62:35173 was elected leader, sending a full tablet report...
I20260808 22:58:58.187402 22409 log.cc:824] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:58.187603 22406 raft_consensus.cc:348] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.187798 22406 raft_consensus.cc:374] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:58.187832 22406 raft_consensus.cc:729] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a0b8eebfad46bcb0f8cdd5a64e37f1, State: Initialized, Role: FOLLOWER
I20260808 22:58:58.187968 22406 consensus_queue.cc:260] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.188259 22406 ts_tablet_manager.cc:1428] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 22:58:58.188272 22409 tablet_bootstrap.cc:492] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: No bootstrap required, opened a new log
I20260808 22:58:58.188280 22112 heartbeater.cc:503] Master 127.1.212.62:35173 was elected leader, sending a full tablet report...
I20260808 22:58:58.188342 22409 ts_tablet_manager.cc:1397] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:58:58.190658 22409 raft_consensus.cc:348] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.190804 22409 raft_consensus.cc:374] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:58.190827 22409 raft_consensus.cc:729] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dbbaca50c34949b79dab2037cc3cb4, State: Initialized, Role: FOLLOWER
I20260808 22:58:58.190948 22409 consensus_queue.cc:260] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.191192 22409 ts_tablet_manager.cc:1428] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:58:58.191290 22390 heartbeater.cc:503] Master 127.1.212.62:35173 was elected leader, sending a full tablet report...
I20260808 22:58:58.201010 22410 raft_consensus.cc:482] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:58:58.201146 22410 raft_consensus.cc:504] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.201630 22410 leader_election.cc:288] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67dbbaca50c34949b79dab2037cc3cb4 (127.1.212.3:43755), b1a0b8eebfad46bcb0f8cdd5a64e37f1 (127.1.212.1:36017)
I20260808 22:58:58.204665 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.201827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:43755 (local address 127.1.212.2:47937)
0808 22:58:58.201930 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.201944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.202000 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.202176 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.202178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.202180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:58.202256 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.202258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.202923 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.202924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.203764 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.203770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.204386 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.204387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.204471 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.204475 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.204541 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.204588 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 22:58:58.204665 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.201900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:47937 (local address 127.1.212.3:43755)
0808 22:58:58.201965 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.201968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.201983 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:58:58.202081 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.202083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.202092 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.202170 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:58.202339 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.202823 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.203861 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.204302 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.204385 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.204396 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.204548 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.204590 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:58:58.205094 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.201927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:47143 (local address 127.1.212.1:36017)
0808 22:58:58.201976 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.201979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.202150 (+ 171us) server_negotiation.cc:407] Connection header received
0808 22:58:58.202232 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.202233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.202246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.202306 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:58:58.202497 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.203031 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.204202 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.204619 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.204708 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.204719 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.204945 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.205011 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:58:58.205140 22342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86cf5aee09c64aa3bec96e821a836d84" candidate_uuid: "61ce5db2a4c643b3af508f07bf621246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dbbaca50c34949b79dab2037cc3cb4" is_pre_election: true
I20260808 22:58:58.205351 22342 raft_consensus.cc:2454] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61ce5db2a4c643b3af508f07bf621246 in term 0.
I20260808 22:58:58.205659 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.201839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36017 (local address 127.1.212.2:47143)
0808 22:58:58.202057 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.202068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:58.202161 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:58.202322 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:58.202323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:58.202324 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:58:58.202413 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.202415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.203128 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.203130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.204098 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:58.204101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.204726 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.204728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:58.204850 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.204855 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:58:58.204919 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:58:58.205570 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":3,"thread_start_us":143,"threads_started":1}
I20260808 22:58:58.205654 22135 leader_election.cc:302] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61ce5db2a4c643b3af508f07bf621246, 67dbbaca50c34949b79dab2037cc3cb4; no voters:
I20260808 22:58:58.205811 22410 raft_consensus.cc:2781] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:58:58.205869 22410 raft_consensus.cc:482] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:58:58.205896 22410 raft_consensus.cc:3037] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:58.206106 22065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86cf5aee09c64aa3bec96e821a836d84" candidate_uuid: "61ce5db2a4c643b3af508f07bf621246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" is_pre_election: true
I20260808 22:58:58.206280 22065 raft_consensus.cc:2454] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61ce5db2a4c643b3af508f07bf621246 in term 0.
I20260808 22:58:58.206996 22410 raft_consensus.cc:504] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.207165 22410 leader_election.cc:288] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [CANDIDATE]: Term 1 election: Requested vote from peers 67dbbaca50c34949b79dab2037cc3cb4 (127.1.212.3:43755), b1a0b8eebfad46bcb0f8cdd5a64e37f1 (127.1.212.1:36017)
I20260808 22:58:58.207314 22342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86cf5aee09c64aa3bec96e821a836d84" candidate_uuid: "61ce5db2a4c643b3af508f07bf621246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dbbaca50c34949b79dab2037cc3cb4"
I20260808 22:58:58.207379 22065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86cf5aee09c64aa3bec96e821a836d84" candidate_uuid: "61ce5db2a4c643b3af508f07bf621246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1"
I20260808 22:58:58.207410 22342 raft_consensus.cc:3037] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:58.207499 22065 raft_consensus.cc:3037] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:58.208305 22065 raft_consensus.cc:2454] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61ce5db2a4c643b3af508f07bf621246 in term 1.
I20260808 22:58:58.208292 22342 raft_consensus.cc:2454] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61ce5db2a4c643b3af508f07bf621246 in term 1.
I20260808 22:58:58.208546 22136 leader_election.cc:302] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61ce5db2a4c643b3af508f07bf621246, b1a0b8eebfad46bcb0f8cdd5a64e37f1; no voters:
I20260808 22:58:58.208648 22410 raft_consensus.cc:2781] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:58.208881 22410 raft_consensus.cc:686] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 1 LEADER]: Becoming Leader. State: Replica: 61ce5db2a4c643b3af508f07bf621246, State: Running, Role: LEADER
I20260808 22:58:58.208997 22410 consensus_queue.cc:237] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } }
I20260808 22:58:58.210039 21915 catalog_manager.cc:5495] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61ce5db2a4c643b3af508f07bf621246 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "61ce5db2a4c643b3af508f07bf621246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61ce5db2a4c643b3af508f07bf621246" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40919 } health_report { overall_health: HEALTHY } } }
I20260808 22:58:58.217694 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.215184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.1.212.2:40919)
0808 22:58:58.215261 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:58.215263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:58.215273 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:58:58.215313 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:58.215314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:58.215334 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:58.215430 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:58:58.215572 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.216034 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.216877 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:58.217077 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:58.217159 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:58.217311 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:58:58.217420 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:58:58.217542 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:58.217575 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 22:58:58.217584 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:58:58.219944 22342 raft_consensus.cc:1261] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 1 FOLLOWER]: Refusing update from remote peer 61ce5db2a4c643b3af508f07bf621246: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:58.220322 22065 raft_consensus.cc:1261] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 1 FOLLOWER]: Refusing update from remote peer 61ce5db2a4c643b3af508f07bf621246: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:58:58.220358 22416 consensus_queue.cc:1035] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67dbbaca50c34949b79dab2037cc3cb4" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:58.220624 22416 consensus_queue.cc:1035] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a0b8eebfad46bcb0f8cdd5a64e37f1" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:58:58.223237 22419 mvcc.cc:205] Tried to move back new op lower bound from 7316397826945470464 to 7316397826904981504. Current Snapshot: MvccSnapshot[applied={T|T < 7316397826945470464}]
I20260808 22:58:58.223829 22420 mvcc.cc:205] Tried to move back new op lower bound from 7316397826945470464 to 7316397826904981504. Current Snapshot: MvccSnapshot[applied={T|T < 7316397826945470464}]
I20260808 22:58:58.224301 22421 mvcc.cc:205] Tried to move back new op lower bound from 7316397826945470464 to 7316397826904981504. Current Snapshot: MvccSnapshot[applied={T|T < 7316397826945470464}]
I20260808 22:58:58.226212 21916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:58.226302 21916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:58:58.227063 21916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37d1b79c9eab4ece8049cd84660e5a3a: Sending DeleteTablet for 3 replicas of tablet 86cf5aee09c64aa3bec96e821a836d84
I20260808 22:58:58.227470 22321 tablet_service.cc:1514] Processing DeleteTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:58 UTC) from {username='slave'} at 127.0.0.1:39328
I20260808 22:58:58.227466 22044 tablet_service.cc:1514] Processing DeleteTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:58 UTC) from {username='slave'} at 127.0.0.1:45228
I20260808 22:58:58.227711 22430 tablet_replica.cc:330] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: stopping tablet replica
I20260808 22:58:58.227820 22430 raft_consensus.cc:2229] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:58.227938 22430 raft_consensus.cc:2258] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:58.228047 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21978
I20260808 22:58:58.228118 22431 tablet_replica.cc:330] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1: stopping tablet replica
I20260808 22:58:58.228222 22431 raft_consensus.cc:2229] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:58:58.228318 22431 raft_consensus.cc:2258] T 86cf5aee09c64aa3bec96e821a836d84 P b1a0b8eebfad46bcb0f8cdd5a64e37f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:58.228546 22430 ts_tablet_manager.cc:1905] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:58.230583 22430 ts_tablet_manager.cc:1918] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:58.230690 22430 log.cc:1196] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/86cf5aee09c64aa3bec96e821a836d84
I20260808 22:58:58.230989 22430 ts_tablet_manager.cc:1939] T 86cf5aee09c64aa3bec96e821a836d84 P 67dbbaca50c34949b79dab2037cc3cb4: Deleting consensus metadata
I20260808 22:58:58.231534 21900 catalog_manager.cc:4841] TS 67dbbaca50c34949b79dab2037cc3cb4 (127.1.212.3:43755): tablet 86cf5aee09c64aa3bec96e821a836d84 (table test-table [id=916baca67f2a40139c00abaf72d196df]) successfully deleted
I20260808 22:58:58.232434 22181 tablet_service.cc:1514] Processing DeleteTablet for tablet 86cf5aee09c64aa3bec96e821a836d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:58:58 UTC) from {username='slave'} at 127.0.0.1:59096
I20260808 22:58:58.232703 22432 tablet_replica.cc:330] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: stopping tablet replica
I20260808 22:58:58.232851 22432 raft_consensus.cc:2229] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:58:58.233032 22432 raft_consensus.cc:2258] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:58:58.233490 22432 ts_tablet_manager.cc:1905] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:58:58.235632 22432 ts_tablet_manager.cc:1918] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:58:58.235723 22432 log.cc:1196] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/86cf5aee09c64aa3bec96e821a836d84
I20260808 22:58:58.236006 22432 ts_tablet_manager.cc:1939] T 86cf5aee09c64aa3bec96e821a836d84 P 61ce5db2a4c643b3af508f07bf621246: Deleting consensus metadata
I20260808 22:58:58.236490 21898 catalog_manager.cc:4841] TS 61ce5db2a4c643b3af508f07bf621246 (127.1.212.2:40919): tablet 86cf5aee09c64aa3bec96e821a836d84 (table test-table [id=916baca67f2a40139c00abaf72d196df]) successfully deleted
W20260808 22:58:58.237900 21901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:36017 (error 108)
I20260808 22:58:58.238260 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22115
I20260808 22:58:58.241034 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:58.240866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:36017 (local address 127.0.0.1:45232)
0808 22:58:58.240934 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:58.240991 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:58:58.241200 21901 catalog_manager.cc:4584] TS b1a0b8eebfad46bcb0f8cdd5a64e37f1 (127.1.212.1:36017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86cf5aee09c64aa3bec96e821a836d84: Network error: Client connection negotiation failed: client connection to 127.1.212.1:36017: connect: Connection refused (error 111)
I20260808 22:58:58.246462 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22254
I20260808 22:58:58.256289 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 21881
2026-08-08T22:58:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (682 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:58:58 dist-test-slave-1x4g krb5kdc[22437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:58:58 dist-test-slave-1x4g krb5kdc[22437](info): set up 2 sockets
Aug 08 22:58:58 dist-test-slave-1x4g krb5kdc[22437](info): commencing operation
krb5kdc: starting...
W20260808 22:58:59.337041 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:58:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:58:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:58:59.471879 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34057
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:34879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:59.563797 22451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:59.564081 22451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:59.564136 22451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:59.565528 22451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:58:59.565619 22451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:59.565644 22451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:59.565663 22451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:58:59.565692 22451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:58:59.567361 22451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:59.567703 22451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:59.567972 22451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:59.573173 22456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.573203 22457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.573173 22459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:59.573459 22451 server_base.cc:1034] running on GCE node
I20260808 22:58:59.573729 22451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:59.574062 22451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:59.575217 22451 hybrid_clock.cc:648] HybridClock initialized: now 1786229939575198 us; error 41 us; skew 500 ppm
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:59.579160 22451 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:58:59.579867 22451 webserver.cc:460] Webserver started at http://127.1.212.62:41843/ using document root <none> and password file <none>
I20260808 22:58:59.580125 22451 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:59.580197 22451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:59.580346 22451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:59.582093 22451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "ed74eaa33cf64d9580adb6ecff062131"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "7c5a50538e056f14bdd2e7bc6d83e7de"
server_key_iv: "0e05e6f7254519163845767bd6aed4c9"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.582540 22451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "ed74eaa33cf64d9580adb6ecff062131"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "7c5a50538e056f14bdd2e7bc6d83e7de"
server_key_iv: "0e05e6f7254519163845767bd6aed4c9"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.584594 22451 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260808 22:58:59.585582 22467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:59.585774 22451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:59.585907 22451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "ed74eaa33cf64d9580adb6ecff062131"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "7c5a50538e056f14bdd2e7bc6d83e7de"
server_key_iv: "0e05e6f7254519163845767bd6aed4c9"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.586014 22451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:59.612694 22451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:59.614995 22451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:59.615185 22451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:59.619736 22451 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:34057
I20260808 22:58:59.619794 22521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:34057 every 8 connection(s)
I20260808 22:58:59.620206 22451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:58:59.621301 22522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:58:59.623926 22522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: Bootstrap starting.
I20260808 22:58:59.624581 22522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: Neither blocks nor log segments found. Creating new log.
I20260808 22:58:59.624965 22522 log.cc:824] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: Log is configured to *not* fsync() on all Append() calls
I20260808 22:58:59.625785 22522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: No bootstrap required, opened a new log
I20260808 22:58:59.627523 22522 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } }
I20260808 22:58:59.627698 22522 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:58:59.627723 22522 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed74eaa33cf64d9580adb6ecff062131, State: Initialized, Role: FOLLOWER
I20260808 22:58:59.627882 22522 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } }
I20260808 22:58:59.627977 22522 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:58:59.628003 22522 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:58:59.628042 22522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:58:59.628645 22522 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } }
I20260808 22:58:59.628744 22522 leader_election.cc:302] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed74eaa33cf64d9580adb6ecff062131; no voters:
I20260808 22:58:59.629032 22522 leader_election.cc:288] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:58:59.629017 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22451
I20260808 22:58:59.629060 22525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:58:59.629154 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:58:59.629195 22525 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [term 1 LEADER]: Becoming Leader. State: Replica: ed74eaa33cf64d9580adb6ecff062131, State: Running, Role: LEADER
I20260808 22:58:59.629297 22525 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } }
I20260808 22:58:59.629356 22522 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:58:59.629400 1872 external_mini_cluster.cc:1428] Setting key 56707a79a42f453e97f8cd9647a9cdf4
I20260808 22:58:59.629670 22525 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed74eaa33cf64d9580adb6ecff062131. Latest consensus state: current_term: 1 leader_uuid: "ed74eaa33cf64d9580adb6ecff062131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } } }
I20260808 22:58:59.629747 22525 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:59.629851 22526 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed74eaa33cf64d9580adb6ecff062131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed74eaa33cf64d9580adb6ecff062131" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34057 } } }
I20260808 22:58:59.629951 22526 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131 [sys.catalog]: This master's current role is: LEADER
I20260808 22:58:59.630344 22531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:58:59.631011 22531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:58:59.632812 22531 catalog_manager.cc:1347] Generated new cluster ID: 65662e27e96843dc9a3355822c086449
I20260808 22:58:59.632872 22531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:59.643487 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:59.630175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.1.212.62:34057)
0808 22:58:59.630404 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:59.630465 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:59.630503 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:58:59.630662 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:59.630670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:59.630735 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:59.630900 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:59.631737 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.633185 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.634316 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.634982 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.635088 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:59.638247 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:59.638253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:59.638253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:59.638270 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:59.641346 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:59.641859 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:59.641863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:59.641863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:59.641989 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:59.642222 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:59.642223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:59.642223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:59.642562 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:59.642760 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:59.643041 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 22:58:59.643214 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260808 22:58:59.664933 22531 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:58:59.665544 22531 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:58:59.677645 22531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: Generated new TSK 0
I20260808 22:58:59.677934 22531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
Entry for principal kudu/127.1.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab.
I20260808 22:58:59.712644 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:34879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:59.800243 22553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:59.800484 22553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:59.800513 22553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:59.801723 22553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:59.801770 22553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:59.801798 22553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:58:59.803324 22553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.1.keytab
--principal=kudu/127.1.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:59.803570 22553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:59.803782 22553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:59.809195 22561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.809187 22559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.809183 22558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:59.809398 22553 server_base.cc:1034] running on GCE node
I20260808 22:58:59.809617 22553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:59.809859 22553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:59.811005 22553 hybrid_clock.cc:648] HybridClock initialized: now 1786229939810994 us; error 38 us; skew 500 ppm
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:59.813802 22553 init.cc:377] Logged in from keytab as kudu/127.1.212.1@KRBTEST.COM (short username kudu)
I20260808 22:58:59.814277 22553 webserver.cc:460] Webserver started at http://127.1.212.1:37217/ using document root <none> and password file <none>
I20260808 22:58:59.814543 22553 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:59.814597 22553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:59.814775 22553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:59.815894 22553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "e2de4448e2ad43cea83cdafd0be95859"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "e0edccbee15b7c5e673075b43349adb6"
server_key_iv: "30dc2cf1afcc84550f076dcc04a4d46a"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.816279 22553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "e2de4448e2ad43cea83cdafd0be95859"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "e0edccbee15b7c5e673075b43349adb6"
server_key_iv: "30dc2cf1afcc84550f076dcc04a4d46a"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.817886 22553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:58:59.818761 22568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:59.818977 22553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:58:59.819084 22553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "e2de4448e2ad43cea83cdafd0be95859"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "e0edccbee15b7c5e673075b43349adb6"
server_key_iv: "30dc2cf1afcc84550f076dcc04a4d46a"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.819149 22553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:58:59.843927 22553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:58:59.846086 22553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:58:59.846208 22553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:58:59.846426 22553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:58:59.846796 22553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:58:59.846832 22553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:59.846856 22553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:58:59.846909 22553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:59.853245 22553 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:40497
I20260808 22:58:59.853294 22691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:40497 every 8 connection(s)
I20260808 22:58:59.853685 22553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:58:59.859350 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22553
I20260808 22:58:59.859468 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:58:59.859688 1872 external_mini_cluster.cc:1428] Setting key cac7e694cb7156744d1a5f9e1963879c
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.1@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:58:59.864702 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:59.854573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:59841 (local address 127.1.212.62:34057)
0808 22:58:59.854652 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:58:59.854655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:58:59.854927 (+ 272us) server_negotiation.cc:407] Connection header received
0808 22:58:59.855475 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:58:59.855477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:58:59.855493 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:58:59.855561 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:58:59.856583 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.857204 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.858256 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.858487 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.858545 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:59.861372 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:58:59.861374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:58:59.861374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:58:59.861384 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:58:59.863133 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:59.863618 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:59.863620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:59.863620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:59.863690 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:58:59.863979 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:58:59.863980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:58:59.863980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:58:59.864110 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:58:59.864189 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:58:59.864500 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 22:58:59.864606 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:58:59.865048 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:58:59.854549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34057 (local address 127.1.212.1:59841)
0808 22:58:59.854771 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 22:58:59.854818 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:58:59.855348 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:58:59.855624 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:58:59.855627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:58:59.856270 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:58:59.856484 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:59.856490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.857307 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.857307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:59.858167 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:58:59.858170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:58:59.858572 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:58:59.858573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:58:59.858681 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:58:59.859305 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:58:59.859315 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:58:59.861261 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:58:59.863240 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:59.863251 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:59.863254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:59.863527 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:59.863771 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:58:59.863772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:58:59.863772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:58:59.863900 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:58:59.864192 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:58:59.864201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:58:59.864353 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 22:58:59.864475 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 22:58:59.864660 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260808 22:58:59.865608 22692 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34057
I20260808 22:58:59.865708 22692 heartbeater.cc:464] Registering TS with master...
I20260808 22:58:59.865931 22692 heartbeater.cc:511] Master 127.1.212.62:34057 requested a full tablet report, sending...
I20260808 22:58:59.866542 22484 ts_manager.cc:194] Registered new tserver with Master: e2de4448e2ad43cea83cdafd0be95859 (127.1.212.1:40497)
I20260808 22:58:59.867169 22484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.1@KRBTEST.COM'} at 127.1.212.1:59841
WARNING: no policy specified for kudu/127.1.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
Entry for principal kudu/127.1.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab.
I20260808 22:58:59.891691 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:34879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:58:59.979277 22697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:58:59.979516 22697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:58:59.979542 22697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:58:59.980844 22697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:58:59.980896 22697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:58:59.980926 22697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:58:59.982506 22697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.2.keytab
--principal=kudu/127.1.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:58:59.982749 22697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:58:59.982954 22697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:58:59.988389 22704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.988391 22703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:58:59.988469 22706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:58:59.988574 22697 server_base.cc:1034] running on GCE node
I20260808 22:58:59.988940 22697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:58:59.989235 22697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:58:59.990415 22697 hybrid_clock.cc:648] HybridClock initialized: now 1786229939990409 us; error 39 us; skew 500 ppm
Aug 08 22:58:59 dist-test-slave-1x4g krb5kdc[22437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:58:59.993106 22697 init.cc:377] Logged in from keytab as kudu/127.1.212.2@KRBTEST.COM (short username kudu)
I20260808 22:58:59.993593 22697 webserver.cc:460] Webserver started at http://127.1.212.2:37971/ using document root <none> and password file <none>
I20260808 22:58:59.993855 22697 fs_manager.cc:359] Metadata directory not provided
I20260808 22:58:59.993937 22697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:58:59.994095 22697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:58:59.995227 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "8e09c83032e341edb0349f30b118f6ff"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "f0cd62e2ecce43c9c6e308bf12f5ff73"
server_key_iv: "2a00ed1b28fb0c4771ed3cbd543c52f5"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.995636 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "8e09c83032e341edb0349f30b118f6ff"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "f0cd62e2ecce43c9c6e308bf12f5ff73"
server_key_iv: "2a00ed1b28fb0c4771ed3cbd543c52f5"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.997210 22697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:58:59.998075 22713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:58:59.998265 22697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:58:59.998382 22697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "8e09c83032e341edb0349f30b118f6ff"
format_stamp: "Formatted at 2026-08-08 22:58:59 on dist-test-slave-1x4g"
server_key: "f0cd62e2ecce43c9c6e308bf12f5ff73"
server_key_iv: "2a00ed1b28fb0c4771ed3cbd543c52f5"
server_key_version: "encryptionkey@0"
I20260808 22:58:59.998468 22697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:00.014474 22697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:00.016537 22697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:00.016739 22697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:00.017051 22697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:00.017395 22697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:00.017429 22697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.017453 22697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:00.017503 22697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.024361 22697 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:41527
I20260808 22:59:00.024425 22833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:41527 every 8 connection(s)
I20260808 22:59:00.024766 22697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:00.028411 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22697
I20260808 22:59:00.028553 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:00.028769 1872 external_mini_cluster.cc:1428] Setting key dae748c8c6e469e3ecc9229538dfd559
Aug 08 22:59:00 dist-test-slave-1x4g krb5kdc[22437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.2@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:00.036322 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.025612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:59241 (local address 127.1.212.62:34057)
0808 22:59:00.025705 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.025708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.025932 (+ 224us) server_negotiation.cc:407] Connection header received
0808 22:59:00.026489 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.026490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.026504 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.026566 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.027584 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.028119 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.029692 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.030042 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.030118 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.032893 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.032895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.032896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:00.032905 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.034472 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.035158 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.035159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.035159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.035221 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.035568 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.035568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.035569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.035698 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.035757 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.036168 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.036249 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:59:00.036715 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.025583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34057 (local address 127.1.212.2:59241)
0808 22:59:00.025775 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.025820 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.026359 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.026642 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.026647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.027252 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:00.027475 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.027481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.028237 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.028238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.029592 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.029597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.030139 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.030140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.030301 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.030989 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:00.031001 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:00.032721 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:00.034626 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:00.034641 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:00.034645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:00.035058 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:00.035302 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:00.035303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:00.035303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:00.035487 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:00.035775 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:00.035785 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:00.036011 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.036150 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.036376 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260808 22:59:00.037469 22835 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34057
I20260808 22:59:00.037603 22835 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:00.037807 22835 heartbeater.cc:511] Master 127.1.212.62:34057 requested a full tablet report, sending...
I20260808 22:59:00.038318 22484 ts_manager.cc:194] Registered new tserver with Master: 8e09c83032e341edb0349f30b118f6ff (127.1.212.2:41527)
I20260808 22:59:00.038902 22484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.2@KRBTEST.COM'} at 127.1.212.2:59241
WARNING: no policy specified for kudu/127.1.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
Entry for principal kudu/127.1.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab.
I20260808 22:59:00.060974 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:34879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:00.152590 22841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:00.152926 22841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:00.152961 22841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:00.154228 22841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:00.154273 22841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:00.154302 22841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:59:00.155891 22841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:34879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.3.keytab
--principal=kudu/127.1.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:34057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:00.156137 22841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:00.156368 22841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:00.162144 22847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.162109 22846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.162117 22849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.162606 22841 server_base.cc:1034] running on GCE node
I20260808 22:59:00.162851 22841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:00.163148 22841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:00.164353 22841 hybrid_clock.cc:648] HybridClock initialized: now 1786229940164312 us; error 75 us; skew 500 ppm
Aug 08 22:59:00 dist-test-slave-1x4g krb5kdc[22437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229940, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:00.167582 22841 init.cc:377] Logged in from keytab as kudu/127.1.212.3@KRBTEST.COM (short username kudu)
I20260808 22:59:00.168138 22841 webserver.cc:460] Webserver started at http://127.1.212.3:37619/ using document root <none> and password file <none>
I20260808 22:59:00.168426 22841 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:00.168488 22841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:00.168658 22841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:00.169958 22841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "141be7150c544cc08d00e7e6efb6476a"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "a4b24002d7e03b53f22c6a5f4f8c74f8"
server_key_iv: "1a33320cff4293b6f75060304e40afc0"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.170396 22841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "141be7150c544cc08d00e7e6efb6476a"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "a4b24002d7e03b53f22c6a5f4f8c74f8"
server_key_iv: "1a33320cff4293b6f75060304e40afc0"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.172091 22841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:00.173228 22856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.173516 22841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:00.173635 22841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "141be7150c544cc08d00e7e6efb6476a"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "a4b24002d7e03b53f22c6a5f4f8c74f8"
server_key_iv: "1a33320cff4293b6f75060304e40afc0"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.173727 22841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:00.195668 22841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:00.197918 22841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:00.198105 22841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:00.198372 22841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:00.198762 22841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:00.198800 22841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.198845 22841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:00.198873 22841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.205421 22841 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:39497
I20260808 22:59:00.205471 22976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:39497 every 8 connection(s)
I20260808 22:59:00.206030 22841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:59:00.207939 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 22841
I20260808 22:59:00.208069 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:00.208257 1872 external_mini_cluster.cc:1428] Setting key 8e986a28fdca1179d806407565a65ed2
Aug 08 22:59:00 dist-test-slave-1x4g krb5kdc[22437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229940, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.3@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:00.215936 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.206840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:48787 (local address 127.1.212.62:34057)
0808 22:59:00.206907 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.206909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.207112 (+ 203us) server_negotiation.cc:407] Connection header received
0808 22:59:00.207629 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.207630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.207641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.207704 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.208976 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.209679 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.210730 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.210917 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.210968 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.213184 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.213186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.213186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:00.213193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.214670 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.215090 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.215092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.215093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.215149 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.215375 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.215376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.215376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.215488 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.215541 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.215800 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.215867 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:59:00.216140 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.206811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:34057 (local address 127.1.212.3:48787)
0808 22:59:00.206973 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.207019 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.207535 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.207761 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.207764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.208524 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:00.208898 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.208905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.209779 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.209780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.210650 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.210653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.210996 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.210997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.211109 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.211616 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:00.211625 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:00.213083 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:00.214751 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:00.214758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:00.214761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:00.215020 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:00.215213 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:00.215214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:00.215215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:00.215320 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:00.215546 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:00.215551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:00.215682 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.215770 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.215905 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260808 22:59:00.216715 22977 heartbeater.cc:347] Connected to a master server at 127.1.212.62:34057
I20260808 22:59:00.216869 22977 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:00.217048 22977 heartbeater.cc:511] Master 127.1.212.62:34057 requested a full tablet report, sending...
I20260808 22:59:00.217465 22484 ts_manager.cc:194] Registered new tserver with Master: 141be7150c544cc08d00e7e6efb6476a (127.1.212.3:39497)
I20260808 22:59:00.217813 22484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.212.3@KRBTEST.COM'} at 127.1.212.3:48787
I20260808 22:59:00.220506 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:59:00.228149 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.222031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51062 (local address 127.1.212.62:34057)
0808 22:59:00.222124 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.222126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.222144 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:59:00.222204 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.222205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.222214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.222261 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.222938 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.223438 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.224221 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.224439 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.224509 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.225857 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.225858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.225859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:00.225865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.227161 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.227485 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.227486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.227486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.227540 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:00.227703 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:00.227703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:00.227704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:00.227849 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.227943 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.228020 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.228083 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:59:00.230166 22484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:00.232002 22484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:00.239173 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.234798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.1.212.1:40497)
0808 22:59:00.235060 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.235065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.235077 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:59:00.235123 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.235125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.235143 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.235421 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:00.235611 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.236437 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.238046 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.238692 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.238792 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.238865 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.239031 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.239076 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260808 22:59:00.239161 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.234616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:40497 (local address 127.0.0.1:57902)
0808 22:59:00.234753 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.234798 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.234891 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.235431 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.235434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.235436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:00.235525 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.235527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.236551 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.236552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.237911 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.237915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.238797 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.238798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.238887 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.238903 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.239008 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.239076 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260808 22:59:00.239276 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.234826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41527 (local address 127.0.0.1:37828)
0808 22:59:00.235235 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.235247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.235303 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.235608 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.235609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.235611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:00.235660 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.235661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.237047 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.237048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.238100 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.238103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.239044 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.239045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.239114 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.239118 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.239150 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.239198 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260808 22:59:00.239909 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.235001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39497 (local address 127.0.0.1:54908)
0808 22:59:00.235267 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.235277 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.235324 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.236002 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.236003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.236005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:00.236044 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.236046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.237304 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.237305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.238227 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.238230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.239194 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.239195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.239743 (+ 548us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.239750 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.239804 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.239845 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
I20260808 22:59:00.239905 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.234972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.1.212.2:41527)
0808 22:59:00.235199 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.235204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.235362 (+ 158us) server_negotiation.cc:407] Connection header received
0808 22:59:00.235408 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.235410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.235437 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.235512 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:00.235736 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.236936 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.238197 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.238920 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.239652 (+ 732us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.239727 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.239757 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.239817 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":139,"threads_started":1}
I20260808 22:59:00.240181 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.235243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.1.212.3:39497)
0808 22:59:00.235830 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.235835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.235846 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:59:00.235882 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.235884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.235899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.235980 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:00.236110 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.237179 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.238320 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.239040 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.239104 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.239162 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.240039 (+ 877us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.240097 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":96,"threads_started":1}
I20260808 22:59:00.240156 22620 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 (DEFAULT_TABLE table=test-table [id=72ad5ff202904276b1f19e4820390554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:00.240578 22620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1c070a8a542518a11bc51f0ca2e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:00.240556 22766 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 (DEFAULT_TABLE table=test-table [id=72ad5ff202904276b1f19e4820390554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:00.240895 22766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1c070a8a542518a11bc51f0ca2e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:00.243491 22908 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 (DEFAULT_TABLE table=test-table [id=72ad5ff202904276b1f19e4820390554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:00.243703 22997 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Bootstrap starting.
I20260808 22:59:00.243878 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1c070a8a542518a11bc51f0ca2e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:00.243978 22998 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Bootstrap starting.
I20260808 22:59:00.244369 22997 tablet_bootstrap.cc:654] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:00.244645 22998 tablet_bootstrap.cc:654] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:00.244897 22997 log.cc:824] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:00.245290 22998 log.cc:824] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:00.245705 22997 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: No bootstrap required, opened a new log
I20260808 22:59:00.245772 22997 ts_tablet_manager.cc:1397] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:59:00.246140 22998 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: No bootstrap required, opened a new log
I20260808 22:59:00.246214 22998 ts_tablet_manager.cc:1397] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:59:00.247295 22997 raft_consensus.cc:348] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.247476 22997 raft_consensus.cc:374] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:00.247524 22997 raft_consensus.cc:729] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e09c83032e341edb0349f30b118f6ff, State: Initialized, Role: FOLLOWER
I20260808 22:59:00.247641 22997 consensus_queue.cc:260] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.247967 22997 ts_tablet_manager.cc:1428] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:59:00.247985 22835 heartbeater.cc:503] Master 127.1.212.62:34057 was elected leader, sending a full tablet report...
I20260808 22:59:00.248510 22998 raft_consensus.cc:348] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.248652 22998 raft_consensus.cc:374] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:00.248674 22998 raft_consensus.cc:729] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2de4448e2ad43cea83cdafd0be95859, State: Initialized, Role: FOLLOWER
I20260808 22:59:00.248842 22998 consensus_queue.cc:260] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.248961 23001 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Bootstrap starting.
I20260808 22:59:00.249150 22998 ts_tablet_manager.cc:1428] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 22:59:00.249289 22692 heartbeater.cc:503] Master 127.1.212.62:34057 was elected leader, sending a full tablet report...
I20260808 22:59:00.249667 23001 tablet_bootstrap.cc:654] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:00.250397 23001 log.cc:824] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:00.251190 23001 tablet_bootstrap.cc:492] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: No bootstrap required, opened a new log
I20260808 22:59:00.251279 23001 ts_tablet_manager.cc:1397] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:59:00.253043 23001 raft_consensus.cc:348] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.253225 23001 raft_consensus.cc:374] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:00.253274 23001 raft_consensus.cc:729] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 141be7150c544cc08d00e7e6efb6476a, State: Initialized, Role: FOLLOWER
I20260808 22:59:00.253428 23001 consensus_queue.cc:260] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.253726 23001 ts_tablet_manager.cc:1428] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 22:59:00.253772 22977 heartbeater.cc:503] Master 127.1.212.62:34057 was elected leader, sending a full tablet report...
W20260808 22:59:00.275424 22836 tablet.cc:2369] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:59:00.354547 22693 tablet.cc:2369] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:00.406064 23003 raft_consensus.cc:482] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:00.406215 23003 raft_consensus.cc:504] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.406643 23003 leader_election.cc:288] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e09c83032e341edb0349f30b118f6ff (127.1.212.2:41527), 141be7150c544cc08d00e7e6efb6476a (127.1.212.3:39497)
I20260808 22:59:00.411362 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.407007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:60473 (local address 127.1.212.2:41527)
0808 22:59:00.407071 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.407075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.407119 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:59:00.407242 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.407243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.407259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.407353 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:00.407585 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.408377 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.410273 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.410894 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.410989 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.411004 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.411226 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.411276 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:59:00.411362 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.407299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:44895 (local address 127.1.212.3:39497)
0808 22:59:00.407372 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.407374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.407382 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:00.407424 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.407424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.407435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.407496 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:00.407685 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.408396 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.410273 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.410894 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.410989 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.411004 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.411226 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.411276 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:59:00.411432 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.406897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:41527 (local address 127.1.212.1:60473)
0808 22:59:00.407052 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.407065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.407150 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.407375 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.407376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.407378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:00.407474 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.407476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.408531 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.408533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.410118 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.410124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.411034 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.411034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.411121 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.411129 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.411196 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.411279 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":101,"thread_start_us":105,"threads_started":1}
I20260808 22:59:00.411677 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.407159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:39497 (local address 127.1.212.1:44895)
0808 22:59:00.407285 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.407299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.407359 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.407520 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.407520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.407522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:00.407604 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.407606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.408534 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.408535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.410119 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.410124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.411022 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.411023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.411121 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.411128 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.411196 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.411279 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":168,"thread_start_us":73,"threads_started":1}
I20260808 22:59:00.411926 22787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1c070a8a542518a11bc51f0ca2e81" candidate_uuid: "e2de4448e2ad43cea83cdafd0be95859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e09c83032e341edb0349f30b118f6ff" is_pre_election: true
I20260808 22:59:00.412149 22787 raft_consensus.cc:2454] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2de4448e2ad43cea83cdafd0be95859 in term 0.
I20260808 22:59:00.412109 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1c070a8a542518a11bc51f0ca2e81" candidate_uuid: "e2de4448e2ad43cea83cdafd0be95859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "141be7150c544cc08d00e7e6efb6476a" is_pre_election: true
I20260808 22:59:00.412271 22928 raft_consensus.cc:2454] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2de4448e2ad43cea83cdafd0be95859 in term 0.
I20260808 22:59:00.412456 22573 leader_election.cc:302] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e09c83032e341edb0349f30b118f6ff, e2de4448e2ad43cea83cdafd0be95859; no voters:
I20260808 22:59:00.412606 23003 raft_consensus.cc:2781] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:00.412672 23003 raft_consensus.cc:482] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:00.412700 23003 raft_consensus.cc:3037] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:00.413506 23003 raft_consensus.cc:504] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.413667 23003 leader_election.cc:288] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [CANDIDATE]: Term 1 election: Requested vote from peers 8e09c83032e341edb0349f30b118f6ff (127.1.212.2:41527), 141be7150c544cc08d00e7e6efb6476a (127.1.212.3:39497)
I20260808 22:59:00.413877 22787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1c070a8a542518a11bc51f0ca2e81" candidate_uuid: "e2de4448e2ad43cea83cdafd0be95859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e09c83032e341edb0349f30b118f6ff"
I20260808 22:59:00.413939 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1c070a8a542518a11bc51f0ca2e81" candidate_uuid: "e2de4448e2ad43cea83cdafd0be95859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "141be7150c544cc08d00e7e6efb6476a"
I20260808 22:59:00.414013 22787 raft_consensus.cc:3037] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:00.414019 22928 raft_consensus.cc:3037] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:00.414846 22787 raft_consensus.cc:2454] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2de4448e2ad43cea83cdafd0be95859 in term 1.
I20260808 22:59:00.414848 22928 raft_consensus.cc:2454] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2de4448e2ad43cea83cdafd0be95859 in term 1.
I20260808 22:59:00.415072 22573 leader_election.cc:302] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 141be7150c544cc08d00e7e6efb6476a, e2de4448e2ad43cea83cdafd0be95859; no voters:
I20260808 22:59:00.415174 23003 raft_consensus.cc:2781] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:00.415354 23003 raft_consensus.cc:686] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [term 1 LEADER]: Becoming Leader. State: Replica: e2de4448e2ad43cea83cdafd0be95859, State: Running, Role: LEADER
I20260808 22:59:00.415470 23003 consensus_queue.cc:237] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } }
I20260808 22:59:00.416504 22484 catalog_manager.cc:5495] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2de4448e2ad43cea83cdafd0be95859 (127.1.212.1). New cstate: current_term: 1 leader_uuid: "e2de4448e2ad43cea83cdafd0be95859" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2de4448e2ad43cea83cdafd0be95859" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 40497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:00.424108 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.421103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.1.212.1:40497)
0808 22:59:00.421186 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.421188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.421196 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:00.421240 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.421241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.421285 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.421370 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:00.421539 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.422055 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.423018 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.423252 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.423317 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.423600 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:00.423799 (+ 199us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:00.423902 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.423946 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.423978 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:59:00.426608 22928 raft_consensus.cc:1261] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 1 FOLLOWER]: Refusing update from remote peer e2de4448e2ad43cea83cdafd0be95859: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:00.426608 22787 raft_consensus.cc:1261] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 1 FOLLOWER]: Refusing update from remote peer e2de4448e2ad43cea83cdafd0be95859: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:00.427013 23003 consensus_queue.cc:1035] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [LEADER]: Connected to new peer: Peer: permanent_uuid: "141be7150c544cc08d00e7e6efb6476a" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:00.427287 23008 consensus_queue.cc:1035] T 32b1c070a8a542518a11bc51f0ca2e81 P e2de4448e2ad43cea83cdafd0be95859 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e09c83032e341edb0349f30b118f6ff" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 41527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:00.430796 23014 mvcc.cc:205] Tried to move back new op lower bound from 7316397835983843328 to 7316397835942662144. Current Snapshot: MvccSnapshot[applied={T|T < 7316397835983843328}]
I20260808 22:59:00.433880 22483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:00.434026 22483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:00.434993 22483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed74eaa33cf64d9580adb6ecff062131: Sending DeleteTablet for 3 replicas of tablet 32b1c070a8a542518a11bc51f0ca2e81
I20260808 22:59:00.435456 22766 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:00 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:37828
I20260808 22:59:00.435465 22620 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:00 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:57902
I20260808 22:59:00.435500 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1c070a8a542518a11bc51f0ca2e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:00 UTC) from {username='kudu', principal='kudu/127.1.212.62@KRBTEST.COM'} at 127.0.0.1:54908
I20260808 22:59:00.435844 23022 tablet_replica.cc:330] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: stopping tablet replica
I20260808 22:59:00.435981 23022 raft_consensus.cc:2229] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:00.436101 23022 raft_consensus.cc:2258] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:00.436113 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22553
I20260808 22:59:00.437749 23024 tablet_replica.cc:330] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: stopping tablet replica
I20260808 22:59:00.437909 23024 raft_consensus.cc:2229] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:00.438054 23024 raft_consensus.cc:2258] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:00.438158 23022 ts_tablet_manager.cc:1905] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:00.438730 23024 ts_tablet_manager.cc:1905] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:00.440936 23024 ts_tablet_manager.cc:1918] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:00.441094 23024 log.cc:1196] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/32b1c070a8a542518a11bc51f0ca2e81
I20260808 22:59:00.441406 23024 ts_tablet_manager.cc:1939] T 32b1c070a8a542518a11bc51f0ca2e81 P 8e09c83032e341edb0349f30b118f6ff: Deleting consensus metadata
I20260808 22:59:00.441958 23022 ts_tablet_manager.cc:1918] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:00.442057 23022 log.cc:1196] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/32b1c070a8a542518a11bc51f0ca2e81
I20260808 22:59:00.442045 22471 catalog_manager.cc:4841] TS 8e09c83032e341edb0349f30b118f6ff (127.1.212.2:41527): tablet 32b1c070a8a542518a11bc51f0ca2e81 (table test-table [id=72ad5ff202904276b1f19e4820390554]) successfully deleted
I20260808 22:59:00.442368 23022 ts_tablet_manager.cc:1939] T 32b1c070a8a542518a11bc51f0ca2e81 P 141be7150c544cc08d00e7e6efb6476a: Deleting consensus metadata
I20260808 22:59:00.442910 22471 catalog_manager.cc:4841] TS 141be7150c544cc08d00e7e6efb6476a (127.1.212.3:39497): tablet 32b1c070a8a542518a11bc51f0ca2e81 (table test-table [id=72ad5ff202904276b1f19e4820390554]) successfully deleted
W20260808 22:59:00.446182 22471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:40497 (error 108)
I20260808 22:59:00.446468 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22697
I20260808 22:59:00.447753 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.447558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:40497 (local address 127.0.0.1:57924)
0808 22:59:00.447642 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.447702 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:40497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 22:59:00.447929 22471 catalog_manager.cc:4584] TS e2de4448e2ad43cea83cdafd0be95859 (127.1.212.1:40497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32b1c070a8a542518a11bc51f0ca2e81: Network error: Client connection negotiation failed: client connection to 127.1.212.1:40497: connect: Connection refused (error 111)
I20260808 22:59:00.453928 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22841
I20260808 22:59:00.461835 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 22451
2026-08-08T22:59:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2209 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T22:59:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:00Z Disabled control of system clock
I20260808 22:59:00.498114 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46353
--webserver_interface=127.1.212.62
--webserver_port=0
--builtin_ntp_servers=127.1.212.20:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:46353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:59:00.589213 23029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:00.589493 23029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:00.589517 23029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:00.590854 23029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:00.590909 23029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:00.590929 23029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:00.590942 23029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:00.590955 23029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:00.592911 23029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:46353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:00.593237 23029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:00.593575 23029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:00.599274 23034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.599288 23035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.599481 23029 server_base.cc:1034] running on GCE node
W20260808 22:59:00.599289 23037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.599912 23029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:00.600338 23029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:00.601534 23029 hybrid_clock.cc:648] HybridClock initialized: now 1786229940601523 us; error 62 us; skew 500 ppm
I20260808 22:59:00.603118 23029 webserver.cc:460] Webserver started at http://127.1.212.62:41235/ using document root <none> and password file <none>
I20260808 22:59:00.603434 23029 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:00.603519 23029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:00.603700 23029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:00.605108 23029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "a1d5c800f9c8485ab7488281392a2408"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "65faaad32c3ddca78a57d829b8dbf02b"
server_key_iv: "8be8eae9d880f5c8948682715112f0d9"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.605556 23029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "a1d5c800f9c8485ab7488281392a2408"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "65faaad32c3ddca78a57d829b8dbf02b"
server_key_iv: "8be8eae9d880f5c8948682715112f0d9"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.607265 23029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:00.608247 23043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.608453 23029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:00.608600 23029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "a1d5c800f9c8485ab7488281392a2408"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "65faaad32c3ddca78a57d829b8dbf02b"
server_key_iv: "8be8eae9d880f5c8948682715112f0d9"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.608705 23029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:00.626783 23029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:00.627238 23029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:00.627444 23029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:00.632305 23029 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:46353
I20260808 22:59:00.632349 23099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:46353 every 8 connection(s)
I20260808 22:59:00.632892 23029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:00.634127 23100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:00.635341 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23029
I20260808 22:59:00.635476 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:00.635691 1872 external_mini_cluster.cc:1428] Setting key 4fd080f90617f68da07df20392f1da01
I20260808 22:59:00.638255 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: Bootstrap starting.
I20260808 22:59:00.639084 23100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:00.639451 23100 log.cc:824] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:00.640413 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: No bootstrap required, opened a new log
I20260808 22:59:00.642870 23100 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } }
I20260808 22:59:00.643129 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.636380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.1.212.62:46353)
0808 22:59:00.636628 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.636636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.636667 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:59:00.637236 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.637245 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.637327 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.637481 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.637784 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.638785 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.639971 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.640537 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.640711 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.641813 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.641821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.641822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:00.642157 (+ 335us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.642202 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:00.642219 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.642338 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.642486 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.642593 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260808 22:59:00.643179 23100 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:00.643333 23100 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d5c800f9c8485ab7488281392a2408, State: Initialized, Role: FOLLOWER
I20260808 22:59:00.643489 23100 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } }
I20260808 22:59:00.643585 23100 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:00.643625 23100 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:00.643656 23100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:00.644395 23100 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } }
I20260808 22:59:00.644500 23100 leader_election.cc:302] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1d5c800f9c8485ab7488281392a2408; no voters:
I20260808 22:59:00.644707 23100 leader_election.cc:288] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:00.645017 23106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } } }
I20260808 22:59:00.645131 23100 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:00.645098 23105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:00.645114 23106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:00.645444 23105 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [term 1 LEADER]: Becoming Leader. State: Replica: a1d5c800f9c8485ab7488281392a2408, State: Running, Role: LEADER
I20260808 22:59:00.645759 23105 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } }
I20260808 22:59:00.646687 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1d5c800f9c8485ab7488281392a2408. Latest consensus state: current_term: 1 leader_uuid: "a1d5c800f9c8485ab7488281392a2408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d5c800f9c8485ab7488281392a2408" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46353 } } }
I20260808 22:59:00.647095 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408 [sys.catalog]: This master's current role is: LEADER
W20260808 22:59:00.647254 23119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:00.647315 23119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:00.647372 23121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:00.647578 23121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:00.649389 23121 catalog_manager.cc:1347] Generated new cluster ID: 03da01730a144c43967f5fe0887c042d
I20260808 22:59:00.649468 23121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:00.662743 23121 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:00.663390 23121 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:00.672035 23121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: Generated new TSK 0
I20260808 22:59:00.672276 23121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:00.682163 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.1:0
--local_ip_for_outbound_sockets=127.1.212.1
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--builtin_ntp_servers=127.1.212.20:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:59:00.772230 23124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:00.772480 23124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:00.772503 23124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:00.773888 23124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:00.773947 23124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:00.773976 23124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.1
I20260808 22:59:00.775585 23124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.212.1
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.1
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:00.775908 23124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:00.776280 23124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:00.782984 23133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.782984 23136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.782991 23132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.783017 23124 server_base.cc:1034] running on GCE node
I20260808 22:59:00.783389 23124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:00.783701 23124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:00.784895 23124 hybrid_clock.cc:648] HybridClock initialized: now 1786229940784864 us; error 52 us; skew 500 ppm
I20260808 22:59:00.786513 23124 webserver.cc:460] Webserver started at http://127.1.212.1:42539/ using document root <none> and password file <none>
I20260808 22:59:00.786902 23124 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:00.786988 23124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:00.787173 23124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:00.788435 23124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data/instance:
uuid: "17eb984c709b498583d561625763d890"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "569db16f0bdfb2cf2107cb119444891a"
server_key_iv: "36d5129165feb1bf2fde88010280e9b6"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.788985 23124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance:
uuid: "17eb984c709b498583d561625763d890"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "569db16f0bdfb2cf2107cb119444891a"
server_key_iv: "36d5129165feb1bf2fde88010280e9b6"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.790773 23124 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:59:00.791937 23142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.792357 23124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 22:59:00.792523 23124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal
uuid: "17eb984c709b498583d561625763d890"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "569db16f0bdfb2cf2107cb119444891a"
server_key_iv: "36d5129165feb1bf2fde88010280e9b6"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.792622 23124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:00.820159 23124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:00.820533 23124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:00.820698 23124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:00.821019 23124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:00.821393 23124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:00.821452 23124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.821498 23124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:00.821527 23124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.828004 23124 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.1:46719
I20260808 22:59:00.828131 23263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.1:46719 every 8 connection(s)
I20260808 22:59:00.828549 23124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/data/info.pb
I20260808 22:59:00.829622 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23124
I20260808 22:59:00.829756 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:00.829988 1872 external_mini_cluster.cc:1428] Setting key 7cb79b4521f598e50b2de13bbe6ea330
I20260808 22:59:00.832878 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.2:0
--local_ip_for_outbound_sockets=127.1.212.2
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--builtin_ntp_servers=127.1.212.20:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:59:00.836094 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.829443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.1:56715 (local address 127.1.212.62:46353)
0808 22:59:00.829498 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.829500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.829885 (+ 385us) server_negotiation.cc:407] Connection header received
0808 22:59:00.830665 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.830666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.830684 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.830737 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.831511 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.832118 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.833859 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.834147 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.834219 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.835657 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.835659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.835659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:00.835666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.835690 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:00.835695 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.835769 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.835937 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.835983 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:00.836320 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.829451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46353 (local address 127.1.212.1:56715)
0808 22:59:00.829714 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.829785 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.830551 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.830832 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.830836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.830852 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:00.831398 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.831402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.832251 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.832253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.833760 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.833766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.834306 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.834308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.834485 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.835469 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:59:00.835470 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:00.835520 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:59:00.835558 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:00.835823 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:00.835831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:00.835835 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.835939 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.836073 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260808 22:59:00.837736 23264 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46353
I20260808 22:59:00.837849 23264 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:00.838542 23264 heartbeater.cc:511] Master 127.1.212.62:46353 requested a full tablet report, sending...
I20260808 22:59:00.839203 23062 ts_manager.cc:194] Registered new tserver with Master: 17eb984c709b498583d561625763d890 (127.1.212.1:46719)
I20260808 22:59:00.839876 23062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.1:56715
W20260808 22:59:00.941560 23267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:00.941790 23267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:00.941814 23267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:00.942984 23267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:00.943022 23267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:00.943048 23267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.2
I20260808 22:59:00.944414 23267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.212.2
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.2
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:00.944697 23267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:00.945088 23267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:00.950659 23273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:00.950721 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.950800 23267 server_base.cc:1034] running on GCE node
W20260808 22:59:00.950663 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:00.951086 23267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:00.951351 23267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:00.952500 23267 hybrid_clock.cc:648] HybridClock initialized: now 1786229940952487 us; error 29 us; skew 500 ppm
I20260808 22:59:00.953877 23267 webserver.cc:460] Webserver started at http://127.1.212.2:32809/ using document root <none> and password file <none>
I20260808 22:59:00.954147 23267 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:00.954196 23267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:00.954367 23267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:00.955523 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data/instance:
uuid: "1cde6285afab48b198e1f3597d01d441"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "e37e50f1b1965222ebcd5bd8d057bbb2"
server_key_iv: "049f41a0ccba5c05404962badf259c7a"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.955897 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance:
uuid: "1cde6285afab48b198e1f3597d01d441"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "e37e50f1b1965222ebcd5bd8d057bbb2"
server_key_iv: "049f41a0ccba5c05404962badf259c7a"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.957376 23267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:00.958140 23285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.958374 23267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:00.958487 23267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal
uuid: "1cde6285afab48b198e1f3597d01d441"
format_stamp: "Formatted at 2026-08-08 22:59:00 on dist-test-slave-1x4g"
server_key: "e37e50f1b1965222ebcd5bd8d057bbb2"
server_key_iv: "049f41a0ccba5c05404962badf259c7a"
server_key_version: "encryptionkey@0"
I20260808 22:59:00.958564 23267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:00.975090 23267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:00.975453 23267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:00.975608 23267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:00.975873 23267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:00.976246 23267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:00.976302 23267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.976349 23267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:00.976389 23267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:00.982754 23267 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.2:40877
I20260808 22:59:00.982801 23405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.2:40877 every 8 connection(s)
I20260808 22:59:00.983158 23267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:00.988724 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.983952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:35889 (local address 127.1.212.62:46353)
0808 22:59:00.984017 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:00.984019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:00.984269 (+ 250us) server_negotiation.cc:407] Connection header received
0808 22:59:00.984852 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:00.984853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:00.984867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:00.984944 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:00.985655 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.986164 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.987241 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.987452 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.987522 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.988476 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:00.988477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:00.988478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:00.988483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:00.988499 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:00.988503 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:00.988552 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:00.988624 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:59:00.988658 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:59:00.988996 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:00.983922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46353 (local address 127.1.212.2:35889)
0808 22:59:00.984135 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:00.984179 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:00.984674 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:00.985027 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:00.985031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:00.985040 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:00.985559 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.985563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.986280 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.986281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.987132 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:00.987135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:00.987569 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:00.987570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:00.987709 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:00.988314 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:59:00.988315 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:00.988347 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:59:00.988384 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:00.988552 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:00.988555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:00.988557 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:59:00.988610 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:59:00.988701 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260808 22:59:00.989591 23406 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46353
I20260808 22:59:00.989706 23406 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:00.989903 23406 heartbeater.cc:511] Master 127.1.212.62:46353 requested a full tablet report, sending...
I20260808 22:59:00.990447 23062 ts_manager.cc:194] Registered new tserver with Master: 1cde6285afab48b198e1f3597d01d441 (127.1.212.2:40877)
I20260808 22:59:00.990806 23062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.2:35889
I20260808 22:59:00.991750 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23267
I20260808 22:59:00.991845 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:00.991981 1872 external_mini_cluster.cc:1428] Setting key c9547adb9bbc7808c1e771f2fa7d9198
I20260808 22:59:00.993148 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.212.3:0
--local_ip_for_outbound_sockets=127.1.212.3
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--builtin_ntp_servers=127.1.212.20:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:59:01.083371 23409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:01.083582 23409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:01.083606 23409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:01.084806 23409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:01.084851 23409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:01.084877 23409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.212.3
I20260808 22:59:01.086289 23409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.212.3
--webserver_port=0
--tserver_master_addrs=127.1.212.62:46353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.212.3
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:01.086603 23409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:01.086987 23409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:01.092597 23415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:01.092578 23417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:01.092604 23414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:01.092621 23409 server_base.cc:1034] running on GCE node
I20260808 22:59:01.093056 23409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:01.093353 23409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:01.094529 23409 hybrid_clock.cc:648] HybridClock initialized: now 1786229941094507 us; error 41 us; skew 500 ppm
I20260808 22:59:01.096072 23409 webserver.cc:460] Webserver started at http://127.1.212.3:34887/ using document root <none> and password file <none>
I20260808 22:59:01.096395 23409 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:01.096444 23409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:01.096647 23409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:01.097820 23409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data/instance:
uuid: "0a1ecea961604e3193df493c5e913bc3"
format_stamp: "Formatted at 2026-08-08 22:59:01 on dist-test-slave-1x4g"
server_key: "82831c903fdeb54e451d157ac4df7a80"
server_key_iv: "31fc3fd6d77b1d4186e06a9f6219c5c6"
server_key_version: "encryptionkey@0"
I20260808 22:59:01.098196 23409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance:
uuid: "0a1ecea961604e3193df493c5e913bc3"
format_stamp: "Formatted at 2026-08-08 22:59:01 on dist-test-slave-1x4g"
server_key: "82831c903fdeb54e451d157ac4df7a80"
server_key_iv: "31fc3fd6d77b1d4186e06a9f6219c5c6"
server_key_version: "encryptionkey@0"
I20260808 22:59:01.099582 23409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:01.100435 23423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:01.100659 23409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:01.100759 23409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal
uuid: "0a1ecea961604e3193df493c5e913bc3"
format_stamp: "Formatted at 2026-08-08 22:59:01 on dist-test-slave-1x4g"
server_key: "82831c903fdeb54e451d157ac4df7a80"
server_key_iv: "31fc3fd6d77b1d4186e06a9f6219c5c6"
server_key_version: "encryptionkey@0"
I20260808 22:59:01.100865 23409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:01.120033 23409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:01.120429 23409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:01.120633 23409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:01.120940 23409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:01.121316 23409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:01.121351 23409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:01.121407 23409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:01.121434 23409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:01.127717 23409 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.3:37237
I20260808 22:59:01.127807 23546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.3:37237 every 8 connection(s)
I20260808 22:59:01.128153 23409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/data/info.pb
I20260808 22:59:01.130785 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23409
I20260808 22:59:01.130901 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:01.131114 1872 external_mini_cluster.cc:1428] Setting key a8a936ba15f49f646f373f50eef550aa
I20260808 22:59:01.134230 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.129029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.3:42593 (local address 127.1.212.62:46353)
0808 22:59:01.129105 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.129107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.129378 (+ 271us) server_negotiation.cc:407] Connection header received
0808 22:59:01.129945 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.129947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.129960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.130033 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:01.130916 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.131581 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.132654 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.132899 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.132977 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.133928 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:01.133930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:01.133930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:01.133935 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:01.133951 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:01.133955 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:01.134006 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.134117 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.134149 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:59:01.134485 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.129039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:46353 (local address 127.1.212.3:42593)
0808 22:59:01.129255 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.129292 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.129835 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.130124 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.130127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.130136 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:01.130777 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.130784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.131695 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.131696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.132554 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.132557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.133006 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.133007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.133151 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.133766 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:59:01.133767 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:01.133797 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:59:01.133822 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:01.134013 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:01.134019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:01.134023 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.134117 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.134231 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260808 22:59:01.135035 23547 heartbeater.cc:347] Connected to a master server at 127.1.212.62:46353
I20260808 22:59:01.135144 23547 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:01.135349 23547 heartbeater.cc:511] Master 127.1.212.62:46353 requested a full tablet report, sending...
I20260808 22:59:01.135761 23062 ts_manager.cc:194] Registered new tserver with Master: 0a1ecea961604e3193df493c5e913bc3 (127.1.212.3:37237)
I20260808 22:59:01.136111 23062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.212.3:42593
I20260808 22:59:01.143217 1872 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:59:01.148099 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.144661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.1.212.62:46353)
0808 22:59:01.144719 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.144721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.144918 (+ 197us) server_negotiation.cc:407] Connection header received
0808 22:59:01.144966 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.144967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.144978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.145050 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:01.145301 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.146092 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.147007 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.147193 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.147250 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.147843 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:01.147844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:01.147844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:01.147849 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:01.147863 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:01.147866 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:01.147927 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.147964 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.147988 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:01.149713 23062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:01.151419 23062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:01.158478 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.153716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:37237 (local address 127.0.0.1:44854)
0808 22:59:01.154046 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.154088 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.154169 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.154678 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.154678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.154680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:01.154717 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.154718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.155779 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.155781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.157154 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.157158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.158126 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.158127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.158217 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.158233 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.158321 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.158378 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":62,"threads_started":1}
I20260808 22:59:01.158502 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.153823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.1.212.3:37237)
0808 22:59:01.154493 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.154499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.154508 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:01.154547 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.154549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.154564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.154795 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:01.154929 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.155663 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.157264 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.158044 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.158101 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.158174 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.158357 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.158401 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":126,"threads_started":1}
I20260808 22:59:01.159034 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.153712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.2:40877 (local address 127.0.0.1:37662)
0808 22:59:01.154045 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.154088 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.154156 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.154519 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.154523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.154525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:01.154613 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.154615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.156848 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.156849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.157820 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.157823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.158791 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.158792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.158857 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.158861 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.158908 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.158953 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"spinlock_wait_cycles":384,"thread_start_us":76,"threads_started":1}
I20260808 22:59:01.159045 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.153804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.1.212.2:40877)
0808 22:59:01.154321 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.154324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.154336 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:59:01.154381 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.154384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.154400 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.154750 (+ 350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:01.154825 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.156000 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.157944 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.158686 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.158786 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.158882 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.158924 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.158964 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":120,"threads_started":1}
I20260808 22:59:01.159809 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.153939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46719 (local address 127.0.0.1:40412)
0808 22:59:01.154283 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.154292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.154351 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.154658 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.154659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.154661 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:01.154725 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.154728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.156089 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.156090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.157168 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.157170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.159562 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.159563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.159626 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.159631 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.159681 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.159728 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":41,"threads_started":1}
I20260808 22:59:01.159732 23476 tablet_service.cc:1467] Processing CreateTablet for tablet 19f183de3c70418f9cf9a7677d6d896b (DEFAULT_TABLE table=test-table [id=e4e9aad24885420ba9ba2e666c387226]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:01.159755 23336 tablet_service.cc:1467] Processing CreateTablet for tablet 19f183de3c70418f9cf9a7677d6d896b (DEFAULT_TABLE table=test-table [id=e4e9aad24885420ba9ba2e666c387226]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:01.160054 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.154063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.1.212.1:46719)
0808 22:59:01.154262 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.154268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.154338 (+ 70us) server_negotiation.cc:407] Connection header received
0808 22:59:01.154411 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.154413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.154430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.154583 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:01.154816 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.155989 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.158211 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.159354 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.159430 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.159513 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.159926 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.159974 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
I20260808 22:59:01.160133 23336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f183de3c70418f9cf9a7677d6d896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:01.160164 23476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f183de3c70418f9cf9a7677d6d896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:01.160760 23193 tablet_service.cc:1467] Processing CreateTablet for tablet 19f183de3c70418f9cf9a7677d6d896b (DEFAULT_TABLE table=test-table [id=e4e9aad24885420ba9ba2e666c387226]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:01.161199 23193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f183de3c70418f9cf9a7677d6d896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:01.163322 23562 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Bootstrap starting.
I20260808 22:59:01.164453 23562 tablet_bootstrap.cc:654] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:01.164608 23564 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: Bootstrap starting.
I20260808 22:59:01.164923 23562 log.cc:824] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:01.164929 23563 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Bootstrap starting.
I20260808 22:59:01.165373 23564 tablet_bootstrap.cc:654] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:01.165630 23563 tablet_bootstrap.cc:654] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:01.165746 23562 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: No bootstrap required, opened a new log
I20260808 22:59:01.165740 23564 log.cc:824] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:01.165823 23562 ts_tablet_manager.cc:1397] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:59:01.165979 23563 log.cc:824] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:01.166517 23564 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: No bootstrap required, opened a new log
I20260808 22:59:01.166605 23564 ts_tablet_manager.cc:1397] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:01.166733 23563 tablet_bootstrap.cc:492] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: No bootstrap required, opened a new log
I20260808 22:59:01.166823 23563 ts_tablet_manager.cc:1397] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:59:01.168875 23562 raft_consensus.cc:348] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.168967 23563 raft_consensus.cc:348] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.169106 23562 raft_consensus.cc:374] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:01.169129 23563 raft_consensus.cc:374] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:01.169150 23562 raft_consensus.cc:729] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1ecea961604e3193df493c5e913bc3, State: Initialized, Role: FOLLOWER
I20260808 22:59:01.169178 23563 raft_consensus.cc:729] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cde6285afab48b198e1f3597d01d441, State: Initialized, Role: FOLLOWER
I20260808 22:59:01.169296 23562 consensus_queue.cc:260] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.169355 23563 consensus_queue.cc:260] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.169417 23564 raft_consensus.cc:348] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.169590 23564 raft_consensus.cc:374] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:01.169636 23564 raft_consensus.cc:729] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17eb984c709b498583d561625763d890, State: Initialized, Role: FOLLOWER
I20260808 22:59:01.169639 23562 ts_tablet_manager.cc:1428] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 22:59:01.169656 23563 ts_tablet_manager.cc:1428] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 22:59:01.169771 23547 heartbeater.cc:503] Master 127.1.212.62:46353 was elected leader, sending a full tablet report...
I20260808 22:59:01.169777 23406 heartbeater.cc:503] Master 127.1.212.62:46353 was elected leader, sending a full tablet report...
I20260808 22:59:01.169800 23564 consensus_queue.cc:260] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.170106 23564 ts_tablet_manager.cc:1428] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 22:59:01.170141 23264 heartbeater.cc:503] Master 127.1.212.62:46353 was elected leader, sending a full tablet report...
I20260808 22:59:01.198673 23569 raft_consensus.cc:482] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:01.198804 23569 raft_consensus.cc:504] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.199201 23569 leader_election.cc:288] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1ecea961604e3193df493c5e913bc3 (127.1.212.3:37237), 17eb984c709b498583d561625763d890 (127.1.212.1:46719)
I20260808 22:59:01.202325 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.199410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.3:37237 (local address 127.1.212.2:38605)
0808 22:59:01.199476 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.199488 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.199536 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.199698 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.199699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.199700 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:01.199783 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.199785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.200440 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.200441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.201352 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.201354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.202050 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.202051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.202127 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.202132 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.202200 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.202251 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:59:01.202349 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.199477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:38605 (local address 127.1.212.3:37237)
0808 22:59:01.199543 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.199545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.199558 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:59:01.199597 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.199598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.199608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.199692 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:01.199854 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.200344 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.201446 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.201972 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.202031 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.202041 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.202220 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.202263 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:59:01.202525 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.199520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.212.2:32875 (local address 127.1.212.1:46719)
0808 22:59:01.199610 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.199612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.199690 (+ 78us) server_negotiation.cc:407] Connection header received
0808 22:59:01.199781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.199782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.199795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.199867 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:01.200032 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.200496 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.201640 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.202141 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.202208 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.202218 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.202386 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.202434 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:59:01.202724 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.199420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46719 (local address 127.1.212.2:32875)
0808 22:59:01.199603 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.199615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:01.199705 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:01.199873 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:01.199874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:01.199876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:01.199955 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.199956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.200584 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.200585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.201550 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:01.201553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.202230 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.202232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:01.202304 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.202307 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:59:01.202368 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 22:59:01.202646 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260808 22:59:01.202726 23497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f183de3c70418f9cf9a7677d6d896b" candidate_uuid: "1cde6285afab48b198e1f3597d01d441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1ecea961604e3193df493c5e913bc3" is_pre_election: true
I20260808 22:59:01.202903 23497 raft_consensus.cc:2454] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cde6285afab48b198e1f3597d01d441 in term 0.
I20260808 22:59:01.203119 23214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f183de3c70418f9cf9a7677d6d896b" candidate_uuid: "1cde6285afab48b198e1f3597d01d441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17eb984c709b498583d561625763d890" is_pre_election: true
I20260808 22:59:01.203230 23288 leader_election.cc:302] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1ecea961604e3193df493c5e913bc3, 1cde6285afab48b198e1f3597d01d441; no voters:
I20260808 22:59:01.203255 23214 raft_consensus.cc:2454] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cde6285afab48b198e1f3597d01d441 in term 0.
I20260808 22:59:01.203389 23569 raft_consensus.cc:2781] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:01.203451 23569 raft_consensus.cc:482] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:01.203485 23569 raft_consensus.cc:3037] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:01.204352 23569 raft_consensus.cc:504] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.204569 23569 leader_election.cc:288] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a1ecea961604e3193df493c5e913bc3 (127.1.212.3:37237), 17eb984c709b498583d561625763d890 (127.1.212.1:46719)
I20260808 22:59:01.204761 23214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f183de3c70418f9cf9a7677d6d896b" candidate_uuid: "1cde6285afab48b198e1f3597d01d441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17eb984c709b498583d561625763d890"
I20260808 22:59:01.204749 23497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f183de3c70418f9cf9a7677d6d896b" candidate_uuid: "1cde6285afab48b198e1f3597d01d441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1ecea961604e3193df493c5e913bc3"
I20260808 22:59:01.204875 23497 raft_consensus.cc:3037] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:01.204875 23214 raft_consensus.cc:3037] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:01.205875 23497 raft_consensus.cc:2454] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cde6285afab48b198e1f3597d01d441 in term 1.
I20260808 22:59:01.205875 23214 raft_consensus.cc:2454] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cde6285afab48b198e1f3597d01d441 in term 1.
I20260808 22:59:01.206123 23286 leader_election.cc:302] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17eb984c709b498583d561625763d890, 1cde6285afab48b198e1f3597d01d441; no voters:
I20260808 22:59:01.206256 23569 raft_consensus.cc:2781] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:01.206470 23569 raft_consensus.cc:686] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 1 LEADER]: Becoming Leader. State: Replica: 1cde6285afab48b198e1f3597d01d441, State: Running, Role: LEADER
I20260808 22:59:01.206580 23569 consensus_queue.cc:237] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } }
I20260808 22:59:01.207520 23061 catalog_manager.cc:5495] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cde6285afab48b198e1f3597d01d441 (127.1.212.2). New cstate: current_term: 1 leader_uuid: "1cde6285afab48b198e1f3597d01d441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cde6285afab48b198e1f3597d01d441" member_type: VOTER last_known_addr { host: "127.1.212.2" port: 40877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:01.218618 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.216041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.1.212.2:40877)
0808 22:59:01.216117 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:01.216119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:01.216146 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:59:01.216194 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:01.216194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:01.216216 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:01.216294 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:01.216441 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.216989 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.217772 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:01.217971 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:01.218046 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:01.218240 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:01.218351 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:01.218443 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:01.218468 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 22:59:01.218491 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:59:01.220547 23214 raft_consensus.cc:1261] T 19f183de3c70418f9cf9a7677d6d896b P 17eb984c709b498583d561625763d890 [term 1 FOLLOWER]: Refusing update from remote peer 1cde6285afab48b198e1f3597d01d441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:01.220546 23497 raft_consensus.cc:1261] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 1 FOLLOWER]: Refusing update from remote peer 1cde6285afab48b198e1f3597d01d441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:01.220973 23573 consensus_queue.cc:1035] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1ecea961604e3193df493c5e913bc3" member_type: VOTER last_known_addr { host: "127.1.212.3" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:01.221122 23569 consensus_queue.cc:1035] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17eb984c709b498583d561625763d890" member_type: VOTER last_known_addr { host: "127.1.212.1" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:01.223876 23578 mvcc.cc:205] Tried to move back new op lower bound from 7316397839236505600 to 7316397839182860288. Current Snapshot: MvccSnapshot[applied={T|T < 7316397839236505600}]
I20260808 22:59:01.224252 23576 mvcc.cc:205] Tried to move back new op lower bound from 7316397839236505600 to 7316397839182860288. Current Snapshot: MvccSnapshot[applied={T|T < 7316397839236505600 or (T in {7316397839236505600})}]
I20260808 22:59:01.226047 23060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:01.226152 23060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:01.227011 23060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1d5c800f9c8485ab7488281392a2408: Sending DeleteTablet for 3 replicas of tablet 19f183de3c70418f9cf9a7677d6d896b
I20260808 22:59:01.227442 23336 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f183de3c70418f9cf9a7677d6d896b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:01 UTC) from {username='slave'} at 127.0.0.1:37662
I20260808 22:59:01.227471 23476 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f183de3c70418f9cf9a7677d6d896b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:01 UTC) from {username='slave'} at 127.0.0.1:44854
I20260808 22:59:01.227717 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23124
I20260808 22:59:01.227732 23193 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f183de3c70418f9cf9a7677d6d896b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:01 UTC) from {username='slave'} at 127.0.0.1:40412
I20260808 22:59:01.228938 23588 tablet_replica.cc:330] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: stopping tablet replica
I20260808 22:59:01.229067 23588 raft_consensus.cc:2229] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:01.229257 23588 raft_consensus.cc:2258] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:01.229948 23588 ts_tablet_manager.cc:1905] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:01.232136 23588 ts_tablet_manager.cc:1918] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:01.232216 23588 log.cc:1196] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-1/wal/wals/19f183de3c70418f9cf9a7677d6d896b
I20260808 22:59:01.232486 23588 ts_tablet_manager.cc:1939] T 19f183de3c70418f9cf9a7677d6d896b P 1cde6285afab48b198e1f3597d01d441: Deleting consensus metadata
I20260808 22:59:01.232982 23044 catalog_manager.cc:4841] TS 1cde6285afab48b198e1f3597d01d441 (127.1.212.2:40877): tablet 19f183de3c70418f9cf9a7677d6d896b (table test-table [id=e4e9aad24885420ba9ba2e666c387226]) successfully deleted
I20260808 22:59:01.236968 23587 tablet_replica.cc:330] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: stopping tablet replica
I20260808 22:59:01.237152 23587 raft_consensus.cc:2229] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:01.237295 23587 raft_consensus.cc:2258] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:01.237797 23587 ts_tablet_manager.cc:1905] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:59:01.238955 23044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.212.1:46719 (error 108)
I20260808 22:59:01.239145 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23267
I20260808 22:59:01.239611 23587 ts_tablet_manager.cc:1918] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:01.239693 23587 log.cc:1196] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Deleting WAL directory at /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229688278322-1872-0/minicluster-data/ts-2/wal/wals/19f183de3c70418f9cf9a7677d6d896b
I20260808 22:59:01.239926 23587 ts_tablet_manager.cc:1939] T 19f183de3c70418f9cf9a7677d6d896b P 0a1ecea961604e3193df493c5e913bc3: Deleting consensus metadata
I20260808 22:59:01.240535 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:01.239464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.1:46719 (local address 127.0.0.1:40416)
0808 22:59:01.240439 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:01.240496 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":942}
W20260808 22:59:01.240708 23044 catalog_manager.cc:4584] TS 17eb984c709b498583d561625763d890 (127.1.212.1:46719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19f183de3c70418f9cf9a7677d6d896b: Network error: Client connection negotiation failed: client connection to 127.1.212.1:46719: connect: Connection refused (error 111)
I20260808 22:59:01.240731 23047 catalog_manager.cc:4841] TS 0a1ecea961604e3193df493c5e913bc3 (127.1.212.3:37237): tablet 19f183de3c70418f9cf9a7677d6d896b (table test-table [id=e4e9aad24885420ba9ba2e666c387226]) successfully deleted
I20260808 22:59:01.247092 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23409
I20260808 22:59:01.254200 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23029
2026-08-08T22:59:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (788 ms)
[----------] 6 tests from EncryptionPolicyTest (9175 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:01 dist-test-slave-1x4g krb5kdc[23592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:01 dist-test-slave-1x4g krb5kdc[23592](info): set up 2 sockets
Aug 08 22:59:01 dist-test-slave-1x4g krb5kdc[23592](info): commencing operation
krb5kdc: starting...
W20260808 22:59:02.340466 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:02.481330 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:02.570053 23606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:02.570304 23606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:02.570329 23606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:02.571539 23606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:02.571584 23606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:02.571599 23606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:02.571612 23606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:02.571624 23606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:02.573210 23606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:02.573464 23606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:02.573658 23606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:02.578961 23612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:02.578966 23616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:02.578994 23611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:02.579281 23606 server_base.cc:1034] running on GCE node
I20260808 22:59:02.579542 23606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:02.579907 23606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:02.581063 23606 hybrid_clock.cc:648] HybridClock initialized: now 1786229942581046 us; error 32 us; skew 500 ppm
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229942, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:02.589123 23606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:02.589684 23606 webserver.cc:460] Webserver started at http://127.0.0.1:38003/ using document root <none> and password file <none>
I20260808 22:59:02.589941 23606 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:02.589988 23606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:02.590178 23606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:02.591346 23606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "d6493a94cae440f091abbef1b56ba494"
format_stamp: "Formatted at 2026-08-08 22:59:02 on dist-test-slave-1x4g"
server_key: "c8dd1c368949ed9b6da8f7b1d77589fd"
server_key_iv: "a13eae3377e7f33e2424daa3364009df"
server_key_version: "encryptionkey@0"
I20260808 22:59:02.591760 23606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "d6493a94cae440f091abbef1b56ba494"
format_stamp: "Formatted at 2026-08-08 22:59:02 on dist-test-slave-1x4g"
server_key: "c8dd1c368949ed9b6da8f7b1d77589fd"
server_key_iv: "a13eae3377e7f33e2424daa3364009df"
server_key_version: "encryptionkey@0"
I20260808 22:59:02.593341 23606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:02.594195 23623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:02.594499 23606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:02.594632 23606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "d6493a94cae440f091abbef1b56ba494"
format_stamp: "Formatted at 2026-08-08 22:59:02 on dist-test-slave-1x4g"
server_key: "c8dd1c368949ed9b6da8f7b1d77589fd"
server_key_iv: "a13eae3377e7f33e2424daa3364009df"
server_key_version: "encryptionkey@0"
I20260808 22:59:02.594723 23606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:02.610456 23606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:02.613018 23606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:02.613188 23606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:02.618050 23606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37635
I20260808 22:59:02.618126 23677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37635 every 8 connection(s)
I20260808 22:59:02.618537 23606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:02.619768 23678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:02.622411 23678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494: Bootstrap starting.
I20260808 22:59:02.623129 23678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:02.623472 23678 log.cc:824] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:02.624336 23678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494: No bootstrap required, opened a new log
I20260808 22:59:02.626184 23678 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260808 22:59:02.626431 23678 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:02.626462 23678 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6493a94cae440f091abbef1b56ba494, State: Initialized, Role: FOLLOWER
I20260808 22:59:02.626629 23678 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260808 22:59:02.626717 23678 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:02.626763 23678 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:02.626820 23678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:02.627488 23678 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260808 22:59:02.627594 23678 leader_election.cc:302] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6493a94cae440f091abbef1b56ba494; no voters:
I20260808 22:59:02.627851 23678 leader_election.cc:288] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:02.627912 23681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:02.628162 23681 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [term 1 LEADER]: Becoming Leader. State: Replica: d6493a94cae440f091abbef1b56ba494, State: Running, Role: LEADER
I20260808 22:59:02.628187 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23606
I20260808 22:59:02.628199 23678 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:02.628306 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:02.628294 23681 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260808 22:59:02.628536 1872 external_mini_cluster.cc:1428] Setting key e2f7361ca363c7b14782dd9bfd5fa3d7
I20260808 22:59:02.628746 23681 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6493a94cae440f091abbef1b56ba494. Latest consensus state: current_term: 1 leader_uuid: "d6493a94cae440f091abbef1b56ba494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260808 22:59:02.628855 23681 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:02.628976 23682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6493a94cae440f091abbef1b56ba494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6493a94cae440f091abbef1b56ba494" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260808 22:59:02.629060 23682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:02.629101 23685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:02.629747 23685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:02.631517 23685 catalog_manager.cc:1347] Generated new cluster ID: b43f86478c084c72a0515123b4c76fcd
I20260808 22:59:02.631592 23685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:59:02.640549 23685 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:02.641332 23685 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:02.641661 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:02.629328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35179 (local address 127.0.0.1:37635)
0808 22:59:02.630688 (+ 1360us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:02.630700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:02.630747 (+ 47us) server_negotiation.cc:407] Connection header received
0808 22:59:02.630931 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:02.630939 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:02.631029 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:02.631197 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:02.632042 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:02.633125 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:02.633944 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:02.634438 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:02.634573 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:02.637518 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:02.637525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:02.637525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:02.637550 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:02.639435 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:02.639933 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:02.639940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:02.639940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:02.640074 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:02.640397 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:02.640398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:02.640399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:02.640751 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:02.640988 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:02.641100 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 22:59:02.641225 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":72,"threads_started":1}
I20260808 22:59:02.642807 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:02.652532 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:02.644386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39965 (local address 127.0.0.1:37635)
0808 22:59:02.644430 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:02.644432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:02.645009 (+ 577us) server_negotiation.cc:407] Connection header received
0808 22:59:02.645094 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:02.645094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:02.645106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:02.645157 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:02.645764 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:02.646311 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:02.647067 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:02.647298 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:02.647491 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:02.649396 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:02.649398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:02.649399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:02.649407 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:02.651075 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:02.651533 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:02.651535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:02.651536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:02.651606 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:02.651875 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:02.651876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:02.651877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:02.652023 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:02.652149 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:02.652306 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:59:02.652385 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:59:02.656863 23685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6493a94cae440f091abbef1b56ba494: Generated new TSK 0
I20260808 22:59:02.657137 23685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:02.660832 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23606
W20260808 22:59:02.668079 23590 connection.cc:535] client connection to 127.0.0.1:37635 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37635): Connection reset by peer (error 104)
2026-08-08T22:59:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23709](info): set up 2 sockets
Aug 08 22:59:02 dist-test-slave-1x4g krb5kdc[23709](info): commencing operation
krb5kdc: starting...
W20260808 22:59:03.748708 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:03 dist-test-slave-1x4g krb5kdc[23709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:03.884667 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:03.971189 23723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:03.971428 23723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:03.971455 23723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:03.972643 23723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:03.972692 23723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:03.972707 23723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:03.972719 23723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:03.972733 23723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:03.974339 23723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43471
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:03.974578 23723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:03.974773 23723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:03.980059 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:03.980151 23728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:03.980234 23723 server_base.cc:1034] running on GCE node
W20260808 22:59:03.980255 23729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:03.980599 23723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:03.980938 23723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:03.982105 23723 hybrid_clock.cc:648] HybridClock initialized: now 1786229943982087 us; error 40 us; skew 500 ppm
Aug 08 22:59:03 dist-test-slave-1x4g krb5kdc[23709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229943, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:03.984723 23723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:03.985298 23723 webserver.cc:460] Webserver started at http://127.0.0.1:40275/ using document root <none> and password file <none>
I20260808 22:59:03.985570 23723 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:03.985642 23723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:03.985797 23723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:03.986958 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "3f5f1f1957614318a8f5d2a1edbbf42d"
format_stamp: "Formatted at 2026-08-08 22:59:03 on dist-test-slave-1x4g"
server_key: "ebcbf89ec2d9b818e8b9045dbcbcd8bd"
server_key_iv: "492fa4fe7e408b55fb1c1880ea728252"
server_key_version: "encryptionkey@0"
I20260808 22:59:03.987357 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "3f5f1f1957614318a8f5d2a1edbbf42d"
format_stamp: "Formatted at 2026-08-08 22:59:03 on dist-test-slave-1x4g"
server_key: "ebcbf89ec2d9b818e8b9045dbcbcd8bd"
server_key_iv: "492fa4fe7e408b55fb1c1880ea728252"
server_key_version: "encryptionkey@0"
I20260808 22:59:03.988857 23723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260808 22:59:03.989694 23739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:03.989903 23723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:03.990029 23723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "3f5f1f1957614318a8f5d2a1edbbf42d"
format_stamp: "Formatted at 2026-08-08 22:59:03 on dist-test-slave-1x4g"
server_key: "ebcbf89ec2d9b818e8b9045dbcbcd8bd"
server_key_iv: "492fa4fe7e408b55fb1c1880ea728252"
server_key_version: "encryptionkey@0"
I20260808 22:59:03.990111 23723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:04.006280 23723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:04.008225 23723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:04.008399 23723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:04.012954 23723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43471
I20260808 22:59:04.013010 23794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43471 every 8 connection(s)
I20260808 22:59:04.013409 23723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:04.014503 23795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:04.016912 23795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: Bootstrap starting.
I20260808 22:59:04.017488 23795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:04.017740 23795 log.cc:824] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:04.018471 23795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: No bootstrap required, opened a new log
I20260808 22:59:04.020058 23795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } }
I20260808 22:59:04.020213 23795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:04.020236 23795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5f1f1957614318a8f5d2a1edbbf42d, State: Initialized, Role: FOLLOWER
I20260808 22:59:04.020344 23795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } }
I20260808 22:59:04.020397 23795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:04.020417 23795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:04.020444 23795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:04.020895 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23723
I20260808 22:59:04.021030 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:04.021252 1872 external_mini_cluster.cc:1428] Setting key c1e1d2b4e8f39232c2932e779696f297
I20260808 22:59:04.021293 23795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } }
I20260808 22:59:04.021417 23795 leader_election.cc:302] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f5f1f1957614318a8f5d2a1edbbf42d; no voters:
I20260808 22:59:04.021653 23795 leader_election.cc:288] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:04.021694 23799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:04.021914 23799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [term 1 LEADER]: Becoming Leader. State: Replica: 3f5f1f1957614318a8f5d2a1edbbf42d, State: Running, Role: LEADER
I20260808 22:59:04.022120 23799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } }
I20260808 22:59:04.021981 23795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:04.023569 23803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } } }
I20260808 22:59:04.023706 23803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:04.023932 23801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5f1f1957614318a8f5d2a1edbbf42d. Latest consensus state: current_term: 1 leader_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5f1f1957614318a8f5d2a1edbbf42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43471 } } }
I20260808 22:59:04.024029 23801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d [sys.catalog]: This master's current role is: LEADER
W20260808 22:59:04.024581 23818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:04.024641 23818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:04.024766 23815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:04.024956 23815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:04.026636 23815 catalog_manager.cc:1347] Generated new cluster ID: 1215cc4d201d41ea9ebdff598003fbff
I20260808 22:59:04.026695 23815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:04.030197 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:04.021795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44807 (local address 127.0.0.1:43471)
0808 22:59:04.022058 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:04.022069 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:04.022107 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:59:04.022229 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:04.022237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:04.022295 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:04.022443 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:04.022658 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:04.024711 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:04.025544 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:04.026091 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:04.026198 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:04.029124 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:04.029129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:04.029129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:04.029374 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:04.029423 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:04.029439 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:04.029609 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:04.029745 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:59:04.029782 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260808 22:59:04.031268 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:04.036464 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:04.032769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46993 (local address 127.0.0.1:43471)
0808 22:59:04.032841 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:04.032843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:04.032969 (+ 126us) server_negotiation.cc:407] Connection header received
0808 22:59:04.033007 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:04.033008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:04.033020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:04.033087 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:04.033225 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:04.033992 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:04.035004 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:04.035349 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:04.035425 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:04.036195 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:04.036197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:04.036197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:04.036203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:04.036222 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:04.036227 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:04.036292 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:04.036359 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 22:59:04.036375 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:59:04.056378 23815 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:04.057273 23815 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:04.063587 23815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f5f1f1957614318a8f5d2a1edbbf42d: Generated new TSK 0
I20260808 22:59:04.063834 23815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:04.070559 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23723
W20260808 22:59:04.078307 23707 connection.cc:535] client connection to 127.0.0.1:43471 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43471): Connection reset by peer (error 104)
2026-08-08T22:59:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:04 dist-test-slave-1x4g krb5kdc[23828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:04 dist-test-slave-1x4g krb5kdc[23828](info): set up 2 sockets
Aug 08 22:59:04 dist-test-slave-1x4g krb5kdc[23828](info): commencing operation
krb5kdc: starting...
W20260808 22:59:05.153353 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:05 dist-test-slave-1x4g krb5kdc[23828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:05.291805 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:05.380338 23842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:05.380594 23842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:05.380621 23842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:05.382015 23842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:05.382095 23842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:05.382112 23842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:05.382125 23842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:05.382138 23842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:05.384349 23842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:05.384724 23842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:05.385073 23842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:05.390568 23848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:05.390581 23850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:05.390568 23847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:05.390885 23842 server_base.cc:1034] running on GCE node
I20260808 22:59:05.391160 23842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:05.391495 23842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:05.392675 23842 hybrid_clock.cc:648] HybridClock initialized: now 1786229945392665 us; error 32 us; skew 500 ppm
Aug 08 22:59:05 dist-test-slave-1x4g krb5kdc[23828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229945, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:05.395617 23842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:05.396193 23842 webserver.cc:460] Webserver started at http://127.0.0.1:42855/ using document root <none> and password file <none>
I20260808 22:59:05.396467 23842 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:05.396524 23842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:05.396704 23842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:05.397894 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "d805bd2ba21244cabb32f34c7ec7055b"
format_stamp: "Formatted at 2026-08-08 22:59:05 on dist-test-slave-1x4g"
server_key: "be7bde873adbcdda01df5bcf71c2293d"
server_key_iv: "bb5ff3d83be005e81d10a476ee7eeca4"
server_key_version: "encryptionkey@0"
I20260808 22:59:05.398319 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "d805bd2ba21244cabb32f34c7ec7055b"
format_stamp: "Formatted at 2026-08-08 22:59:05 on dist-test-slave-1x4g"
server_key: "be7bde873adbcdda01df5bcf71c2293d"
server_key_iv: "bb5ff3d83be005e81d10a476ee7eeca4"
server_key_version: "encryptionkey@0"
I20260808 22:59:05.399827 23842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:05.400693 23859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:05.400913 23842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:05.401034 23842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "d805bd2ba21244cabb32f34c7ec7055b"
format_stamp: "Formatted at 2026-08-08 22:59:05 on dist-test-slave-1x4g"
server_key: "be7bde873adbcdda01df5bcf71c2293d"
server_key_iv: "bb5ff3d83be005e81d10a476ee7eeca4"
server_key_version: "encryptionkey@0"
I20260808 22:59:05.401124 23842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:05.404902 23842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:05.407691 23842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:05.407905 23842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:05.412451 23842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45341
I20260808 22:59:05.412492 23914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45341 every 8 connection(s)
I20260808 22:59:05.412954 23842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:05.414150 23915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:05.416895 23915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: Bootstrap starting.
I20260808 22:59:05.417598 23915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:05.417946 23915 log.cc:824] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:05.418473 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23842
I20260808 22:59:05.418599 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:05.418795 1872 external_mini_cluster.cc:1428] Setting key 9451f4ad10f1e7f02bf571e55be80317
I20260808 22:59:05.419056 23915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: No bootstrap required, opened a new log
I20260808 22:59:05.421496 23915 raft_consensus.cc:348] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } }
I20260808 22:59:05.421667 23915 raft_consensus.cc:374] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:05.421690 23915 raft_consensus.cc:729] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d805bd2ba21244cabb32f34c7ec7055b, State: Initialized, Role: FOLLOWER
I20260808 22:59:05.422118 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:05.419449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50581 (local address 127.0.0.1:45341)
0808 22:59:05.419678 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:05.419690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:05.419813 (+ 123us) server_negotiation.cc:407] Connection header received
0808 22:59:05.420455 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:05.420463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:05.420494 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:05.420661 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:05.421613 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:05.421617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:05.421618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:05.421623 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:05.421692 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:05.421709 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:05.421785 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:05.421898 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 22:59:05.421948 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":120,"threads_started":1}
I20260808 22:59:05.421838 23915 consensus_queue.cc:260] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } }
I20260808 22:59:05.422302 23915 raft_consensus.cc:388] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:05.422358 23915 raft_consensus.cc:482] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:05.422413 23915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:05.423202 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:05.423337 23915 raft_consensus.cc:504] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } }
I20260808 22:59:05.423477 23915 leader_election.cc:302] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d805bd2ba21244cabb32f34c7ec7055b; no voters:
I20260808 22:59:05.423697 23915 leader_election.cc:288] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:05.423727 23920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:05.423874 23920 raft_consensus.cc:686] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [term 1 LEADER]: Becoming Leader. State: Replica: d805bd2ba21244cabb32f34c7ec7055b, State: Running, Role: LEADER
I20260808 22:59:05.423971 23920 consensus_queue.cc:237] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } }
I20260808 22:59:05.424016 23915 sys_catalog.cc:564] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:05.424368 23920 sys_catalog.cc:455] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d805bd2ba21244cabb32f34c7ec7055b. Latest consensus state: current_term: 1 leader_uuid: "d805bd2ba21244cabb32f34c7ec7055b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } } }
I20260808 22:59:05.424491 23920 sys_catalog.cc:458] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:05.424384 23921 sys_catalog.cc:455] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d805bd2ba21244cabb32f34c7ec7055b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d805bd2ba21244cabb32f34c7ec7055b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45341 } } }
I20260808 22:59:05.424669 23921 sys_catalog.cc:458] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:05.426415 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:05.425020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48607 (local address 127.0.0.1:45341)
0808 22:59:05.425066 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:05.425068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:05.425289 (+ 221us) server_negotiation.cc:407] Connection header received
0808 22:59:05.425360 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:05.425361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:05.425368 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:05.425421 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:05.426150 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:05.426151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:05.426152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:05.426156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:05.426172 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:05.426177 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:05.426230 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:05.426307 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 22:59:05.426324 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 22:59:05.426424 23941 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:05.426831 23941 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:05.426899 23932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:05.427093 23932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:05.428848 23932 catalog_manager.cc:1347] Generated new cluster ID: 0b471c4e150e44beb599c544f558fc7a
I20260808 22:59:05.428918 23932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:05.449029 23932 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:05.449262 23932 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:05.463279 23932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d805bd2ba21244cabb32f34c7ec7055b: Generated new TSK 0
I20260808 22:59:05.463570 23932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:05.474998 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23842
2026-08-08T22:59:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:05 dist-test-slave-1x4g krb5kdc[23947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:05 dist-test-slave-1x4g krb5kdc[23947](info): set up 2 sockets
Aug 08 22:59:05 dist-test-slave-1x4g krb5kdc[23947](info): commencing operation
krb5kdc: starting...
W20260808 22:59:06.553383 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[23947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:06.689692 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:06.777616 23964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:06.777863 23964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:06.777889 23964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:06.779103 23964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:06.779147 23964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:06.779161 23964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:06.779174 23964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:06.779187 23964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:06.780706 23964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:06.780978 23964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:06.781191 23964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:06.786329 23972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:06.786388 23964 server_base.cc:1034] running on GCE node
W20260808 22:59:06.786339 23969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:06.786350 23970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:06.786854 23964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:06.787216 23964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:06.788388 23964 hybrid_clock.cc:648] HybridClock initialized: now 1786229946788373 us; error 43 us; skew 500 ppm
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[23947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:06.791020 23964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:06.791436 23964 webserver.cc:460] Webserver started at http://127.0.0.1:43553/ using document root <none> and password file <none>
I20260808 22:59:06.791637 23964 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:06.791680 23964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:06.791774 23964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:06.792914 23964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "151aafd357794c7e99523df7dbbe14df"
format_stamp: "Formatted at 2026-08-08 22:59:06 on dist-test-slave-1x4g"
server_key: "36d81b7fd982e6a9b53a0762ff39b3de"
server_key_iv: "8e06f8d29488b340dc36a5b1d89cbacc"
server_key_version: "encryptionkey@0"
I20260808 22:59:06.793253 23964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "151aafd357794c7e99523df7dbbe14df"
format_stamp: "Formatted at 2026-08-08 22:59:06 on dist-test-slave-1x4g"
server_key: "36d81b7fd982e6a9b53a0762ff39b3de"
server_key_iv: "8e06f8d29488b340dc36a5b1d89cbacc"
server_key_version: "encryptionkey@0"
I20260808 22:59:06.794595 23964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:06.795354 23979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:06.795531 23964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:06.795630 23964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "151aafd357794c7e99523df7dbbe14df"
format_stamp: "Formatted at 2026-08-08 22:59:06 on dist-test-slave-1x4g"
server_key: "36d81b7fd982e6a9b53a0762ff39b3de"
server_key_iv: "8e06f8d29488b340dc36a5b1d89cbacc"
server_key_version: "encryptionkey@0"
I20260808 22:59:06.795693 23964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:06.809195 23964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:06.811545 23964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:06.811691 23964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:06.816027 23964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37639
I20260808 22:59:06.816071 24035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37639 every 8 connection(s)
I20260808 22:59:06.816469 23964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:06.817601 24036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:06.820132 24036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df: Bootstrap starting.
I20260808 22:59:06.820873 24036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:06.821195 24036 log.cc:824] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:06.822058 24036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df: No bootstrap required, opened a new log
I20260808 22:59:06.823781 24036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260808 22:59:06.823966 24036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:06.824025 24036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151aafd357794c7e99523df7dbbe14df, State: Initialized, Role: FOLLOWER
I20260808 22:59:06.824203 24036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260808 22:59:06.824288 24036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:06.824330 24036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:06.824383 24036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:06.825091 24036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260808 22:59:06.825217 24036 leader_election.cc:302] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151aafd357794c7e99523df7dbbe14df; no voters:
I20260808 22:59:06.825429 24036 leader_election.cc:288] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:06.825474 24039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:06.825702 24039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [term 1 LEADER]: Becoming Leader. State: Replica: 151aafd357794c7e99523df7dbbe14df, State: Running, Role: LEADER
I20260808 22:59:06.825717 24036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:06.825812 24039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260808 22:59:06.826248 24040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151aafd357794c7e99523df7dbbe14df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260808 22:59:06.826340 24040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:06.826381 24041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151aafd357794c7e99523df7dbbe14df. Latest consensus state: current_term: 1 leader_uuid: "151aafd357794c7e99523df7dbbe14df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151aafd357794c7e99523df7dbbe14df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260808 22:59:06.826460 24041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:06.826730 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 23964
I20260808 22:59:06.826798 24049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:06.826862 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:06.827050 1872 external_mini_cluster.cc:1428] Setting key 1cf23155f3a8cc839f102d48d51399f4
I20260808 22:59:06.827384 24049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:06.830366 24049 catalog_manager.cc:1347] Generated new cluster ID: ada2e8f5fa064e989f0209631fb8ff8a
I20260808 22:59:06.830495 24049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[23947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:59:06.838922 24049 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:06.839710 24049 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:06.842170 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:06.827724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58031 (local address 127.0.0.1:37639)
0808 22:59:06.828127 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:06.828139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:06.828182 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:59:06.828310 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:06.828317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:06.828387 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:06.828517 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:06.829815 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:06.831278 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:06.832373 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:06.833051 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:06.833160 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:06.837806 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:06.837812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:06.837813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:06.837829 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:06.840214 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:06.840694 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:06.840697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:06.840698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:06.840841 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:06.841066 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:06.841067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:06.841068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:06.841356 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:06.841513 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:06.841639 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 22:59:06.841901 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":77,"threads_started":1}
I20260808 22:59:06.842975 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:06.847661 24049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 151aafd357794c7e99523df7dbbe14df: Generated new TSK 0
I20260808 22:59:06.847884 24049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:06.851992 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:06.844525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36839 (local address 127.0.0.1:37639)
0808 22:59:06.844558 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:06.844560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:06.844670 (+ 110us) server_negotiation.cc:407] Connection header received
0808 22:59:06.844732 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:06.844733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:06.844742 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:06.844839 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:06.845551 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:06.846367 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:06.847553 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:06.847852 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:06.847930 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:06.849427 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:06.849428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:06.849429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:06.849436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:06.850869 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:06.851269 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:06.851270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:06.851270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:06.851323 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:06.851531 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:06.851531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:06.851532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:06.851639 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:06.851712 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:06.851853 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 22:59:06.851927 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 22:59:06.854143 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 23964
2026-08-08T22:59:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[24068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[24068](info): set up 2 sockets
Aug 08 22:59:06 dist-test-slave-1x4g krb5kdc[24068](info): commencing operation
krb5kdc: starting...
W20260808 22:59:07.938017 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:08 dist-test-slave-1x4g krb5kdc[24068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:08.076747 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:08.163178 24081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:08.163424 24081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:08.163453 24081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:08.164745 24081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:08.164826 24081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:08.164844 24081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:08.164857 24081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:08.164870 24081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:08.166457 24081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:08.166709 24081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:08.166898 24081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:08.172247 24086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:08.172277 24081 server_base.cc:1034] running on GCE node
W20260808 22:59:08.172247 24090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:08.172282 24087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:08.172730 24081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:08.173085 24081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:08.174257 24081 hybrid_clock.cc:648] HybridClock initialized: now 1786229948174242 us; error 37 us; skew 500 ppm
Aug 08 22:59:08 dist-test-slave-1x4g krb5kdc[24068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229948, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:08.182345 24081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:08.182906 24081 webserver.cc:460] Webserver started at http://127.0.0.1:40475/ using document root <none> and password file <none>
I20260808 22:59:08.183183 24081 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:08.183254 24081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:08.183408 24081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:08.184569 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f09518c21d0343cf9e9b766e7d8a5600"
format_stamp: "Formatted at 2026-08-08 22:59:08 on dist-test-slave-1x4g"
server_key: "162fcccb13dfc7689059ce8472e99569"
server_key_iv: "577c5e612a6c043395a6c1c0e7ac6e56"
server_key_version: "encryptionkey@0"
I20260808 22:59:08.184996 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f09518c21d0343cf9e9b766e7d8a5600"
format_stamp: "Formatted at 2026-08-08 22:59:08 on dist-test-slave-1x4g"
server_key: "162fcccb13dfc7689059ce8472e99569"
server_key_iv: "577c5e612a6c043395a6c1c0e7ac6e56"
server_key_version: "encryptionkey@0"
I20260808 22:59:08.186499 24081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:08.187288 24101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:08.187485 24081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:08.187597 24081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f09518c21d0343cf9e9b766e7d8a5600"
format_stamp: "Formatted at 2026-08-08 22:59:08 on dist-test-slave-1x4g"
server_key: "162fcccb13dfc7689059ce8472e99569"
server_key_iv: "577c5e612a6c043395a6c1c0e7ac6e56"
server_key_version: "encryptionkey@0"
I20260808 22:59:08.187677 24081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:08.198668 24081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:08.200850 24081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:08.201025 24081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:08.205564 24081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35609
I20260808 22:59:08.205616 24158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35609 every 8 connection(s)
I20260808 22:59:08.206017 24081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:08.207111 24159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:08.209718 24159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600: Bootstrap starting.
I20260808 22:59:08.210373 24159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:08.210669 24159 log.cc:824] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:08.211457 24159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600: No bootstrap required, opened a new log
I20260808 22:59:08.212975 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24081
I20260808 22:59:08.213095 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:08.213299 1872 external_mini_cluster.cc:1428] Setting key 3c05e6e139f5ed42ba73e4ae58c3bf43
I20260808 22:59:08.213697 24159 raft_consensus.cc:348] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } }
I20260808 22:59:08.213900 24159 raft_consensus.cc:374] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:08.214001 24159 raft_consensus.cc:729] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09518c21d0343cf9e9b766e7d8a5600, State: Initialized, Role: FOLLOWER
I20260808 22:59:08.214162 24159 consensus_queue.cc:260] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } }
I20260808 22:59:08.214265 24159 raft_consensus.cc:388] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:08.214318 24159 raft_consensus.cc:482] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:08.214370 24159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:08.215449 24159 raft_consensus.cc:504] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } }
I20260808 22:59:08.215574 24159 leader_election.cc:302] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f09518c21d0343cf9e9b766e7d8a5600; no voters:
I20260808 22:59:08.215776 24159 leader_election.cc:288] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:08.215807 24165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:08.215931 24165 raft_consensus.cc:686] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [term 1 LEADER]: Becoming Leader. State: Replica: f09518c21d0343cf9e9b766e7d8a5600, State: Running, Role: LEADER
I20260808 22:59:08.216079 24165 consensus_queue.cc:237] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } }
I20260808 22:59:08.216084 24159 sys_catalog.cc:564] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:08.217031 24167 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09518c21d0343cf9e9b766e7d8a5600. Latest consensus state: current_term: 1 leader_uuid: "f09518c21d0343cf9e9b766e7d8a5600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } } }
I20260808 22:59:08.217139 24167 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:08.217451 24176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:08.217869 24166 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f09518c21d0343cf9e9b766e7d8a5600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09518c21d0343cf9e9b766e7d8a5600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35609 } } }
I20260808 22:59:08.217998 24166 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:08.218052 24176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:08.219767 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:08.213908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50381 (local address 127.0.0.1:35609)
0808 22:59:08.214151 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:08.214162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:08.214201 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:59:08.214326 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:08.214333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:08.214394 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:08.214536 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:08.214782 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:08.215797 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:08.217103 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:08.217820 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:08.218343 (+ 523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:08.218857 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:08.218862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:08.218862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:08.219103 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:08.219145 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:08.219162 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:08.219234 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:08.219467 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 22:59:08.219541 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260808 22:59:08.220100 24176 catalog_manager.cc:1347] Generated new cluster ID: fa8289a77ff648dfae20252ba40bd980
I20260808 22:59:08.220165 24176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:08.221004 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:08.226426 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:08.222632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39135 (local address 127.0.0.1:35609)
0808 22:59:08.222676 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:08.222678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:08.222820 (+ 142us) server_negotiation.cc:407] Connection header received
0808 22:59:08.222888 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:08.222889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:08.222923 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:08.222979 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:08.223147 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:08.223861 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:08.224952 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:08.225261 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:08.225339 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:08.226131 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:08.226133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:08.226133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:08.226153 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:08.226171 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:08.226175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:08.226228 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:08.226311 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 22:59:08.226339 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:08.231772 24176 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:08.232367 24176 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:08.243103 24176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f09518c21d0343cf9e9b766e7d8a5600: Generated new TSK 0
I20260808 22:59:08.243360 24176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:08.246312 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24081
2026-08-08T22:59:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:08 dist-test-slave-1x4g krb5kdc[24193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:08 dist-test-slave-1x4g krb5kdc[24193](info): set up 2 sockets
Aug 08 22:59:08 dist-test-slave-1x4g krb5kdc[24193](info): commencing operation
krb5kdc: starting...
W20260808 22:59:09.340528 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:09 dist-test-slave-1x4g krb5kdc[24193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:09.475705 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35501
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:09.561187 24206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:09.561434 24206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:09.561460 24206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:09.562722 24206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:09.562775 24206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:09.562790 24206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:09.562804 24206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:09.562816 24206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:09.564440 24206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:09.564718 24206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:09.564997 24206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:09.570211 24214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:09.570266 24212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:09.570271 24211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:09.570312 24206 server_base.cc:1034] running on GCE node
I20260808 22:59:09.570642 24206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:09.570966 24206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:09.572147 24206 hybrid_clock.cc:648] HybridClock initialized: now 1786229949572138 us; error 35 us; skew 500 ppm
Aug 08 22:59:09 dist-test-slave-1x4g krb5kdc[24193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:09.574834 24206 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:09.575376 24206 webserver.cc:460] Webserver started at http://127.0.0.1:41077/ using document root <none> and password file <none>
I20260808 22:59:09.575619 24206 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:09.575687 24206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:09.575839 24206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:09.577124 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "066090c7e69d450183a0ab4588a029fb"
format_stamp: "Formatted at 2026-08-08 22:59:09 on dist-test-slave-1x4g"
server_key: "c9e6cc6935b320d9c09c90804d989787"
server_key_iv: "d728c34a34095087261c98c87c27556f"
server_key_version: "encryptionkey@0"
I20260808 22:59:09.577503 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "066090c7e69d450183a0ab4588a029fb"
format_stamp: "Formatted at 2026-08-08 22:59:09 on dist-test-slave-1x4g"
server_key: "c9e6cc6935b320d9c09c90804d989787"
server_key_iv: "d728c34a34095087261c98c87c27556f"
server_key_version: "encryptionkey@0"
I20260808 22:59:09.578914 24206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:09.579655 24221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:09.579826 24206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:09.579950 24206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "066090c7e69d450183a0ab4588a029fb"
format_stamp: "Formatted at 2026-08-08 22:59:09 on dist-test-slave-1x4g"
server_key: "c9e6cc6935b320d9c09c90804d989787"
server_key_iv: "d728c34a34095087261c98c87c27556f"
server_key_version: "encryptionkey@0"
I20260808 22:59:09.580029 24206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:09.583604 24206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:09.586372 24206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:09.586541 24206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:09.590991 24206 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35501
I20260808 22:59:09.591061 24281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35501 every 8 connection(s)
I20260808 22:59:09.591419 24206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:09.592051 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24206
I20260808 22:59:09.592216 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:09.592449 1872 external_mini_cluster.cc:1428] Setting key e3cce6431f990af3eab6baaa67b2bdad
I20260808 22:59:09.592506 24282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:09.595168 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:09.593057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34641 (local address 127.0.0.1:35501)
0808 22:59:09.593240 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:09.593249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:09.593286 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:59:09.593809 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:09.593816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:09.593844 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:09.593974 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:09.594682 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:09.594685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:09.594685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:09.594689 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:09.594729 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:09.594739 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:09.594799 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:09.594890 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 22:59:09.594944 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260808 22:59:09.596247 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:09.596535 24282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb: Bootstrap starting.
I20260808 22:59:09.597394 24282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:09.597803 24282 log.cc:824] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:09.598572 24282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb: No bootstrap required, opened a new log
I20260808 22:59:09.599412 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:09.598260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56203 (local address 127.0.0.1:35501)
0808 22:59:09.598299 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:09.598302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:09.598428 (+ 126us) server_negotiation.cc:407] Connection header received
0808 22:59:09.598518 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:09.598519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:09.598524 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:09.598582 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:09.599203 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:09.599205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:09.599205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:09.599210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:09.599223 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:09.599227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:09.599277 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:09.599331 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 22:59:09.599347 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:09.600960 24282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } }
I20260808 22:59:09.601130 24282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:09.601154 24282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066090c7e69d450183a0ab4588a029fb, State: Initialized, Role: FOLLOWER
I20260808 22:59:09.601294 24282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } }
I20260808 22:59:09.601374 24282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:09.601428 24282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:09.601498 24282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:09.602231 24282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } }
I20260808 22:59:09.602352 24282 leader_election.cc:302] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 066090c7e69d450183a0ab4588a029fb; no voters:
I20260808 22:59:09.602572 24282 leader_election.cc:288] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:09.602639 24294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:09.602893 24294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [term 1 LEADER]: Becoming Leader. State: Replica: 066090c7e69d450183a0ab4588a029fb, State: Running, Role: LEADER
I20260808 22:59:09.602936 24282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:09.603006 24294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } }
I20260808 22:59:09.603413 24295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "066090c7e69d450183a0ab4588a029fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } } }
I20260808 22:59:09.603715 24295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:09.603619 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 066090c7e69d450183a0ab4588a029fb. Latest consensus state: current_term: 1 leader_uuid: "066090c7e69d450183a0ab4588a029fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066090c7e69d450183a0ab4588a029fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35501 } } }
I20260808 22:59:09.603896 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:09.604096 24303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:09.604674 24303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:09.606391 24303 catalog_manager.cc:1347] Generated new cluster ID: 6c051f90e034407c910fcf2c1ecdf308
I20260808 22:59:09.606451 24303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:09.614498 24303 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:09.614702 24303 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:09.622366 24303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 066090c7e69d450183a0ab4588a029fb: Generated new TSK 0
I20260808 22:59:09.622608 24303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:09.626151 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24206
2026-08-08T22:59:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:09 dist-test-slave-1x4g krb5kdc[24319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:09 dist-test-slave-1x4g krb5kdc[24319](info): set up 2 sockets
Aug 08 22:59:09 dist-test-slave-1x4g krb5kdc[24319](info): commencing operation
krb5kdc: starting...
W20260808 22:59:10.698093 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:10 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:10.844040 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:10.941948 24333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:10.942240 24333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:10.942329 24333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:10.944243 24333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:10.944303 24333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:10.944324 24333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:10.944342 24333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:10.944412 24333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:10.946938 24333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:10.947263 24333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:10.947526 24333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:10.954360 24338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:10.954424 24341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:10.954424 24339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:10.954747 24333 server_base.cc:1034] running on GCE node
I20260808 22:59:10.955034 24333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:10.955387 24333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:10.956578 24333 hybrid_clock.cc:648] HybridClock initialized: now 1786229950956554 us; error 44 us; skew 500 ppm
Aug 08 22:59:10 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:10.960323 24333 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:10.961021 24333 webserver.cc:460] Webserver started at http://127.0.0.1:40787/ using document root <none> and password file <none>
I20260808 22:59:10.961372 24333 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:10.961452 24333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:10.961633 24333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:10.963332 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "392d2630e7f64c6e8e9e8f097c4694f2"
format_stamp: "Formatted at 2026-08-08 22:59:10 on dist-test-slave-1x4g"
server_key: "eef60b5d6ab2bb833102a9eea0d22023"
server_key_iv: "222a479324e5271338eeda6608019cdb"
server_key_version: "encryptionkey@0"
I20260808 22:59:10.963681 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "392d2630e7f64c6e8e9e8f097c4694f2"
format_stamp: "Formatted at 2026-08-08 22:59:10 on dist-test-slave-1x4g"
server_key: "eef60b5d6ab2bb833102a9eea0d22023"
server_key_iv: "222a479324e5271338eeda6608019cdb"
server_key_version: "encryptionkey@0"
I20260808 22:59:10.965154 24333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 22:59:10.965903 24348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:10.966097 24333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:10.966177 24333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "392d2630e7f64c6e8e9e8f097c4694f2"
format_stamp: "Formatted at 2026-08-08 22:59:10 on dist-test-slave-1x4g"
server_key: "eef60b5d6ab2bb833102a9eea0d22023"
server_key_iv: "222a479324e5271338eeda6608019cdb"
server_key_version: "encryptionkey@0"
I20260808 22:59:10.966238 24333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:10.976709 24333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:10.978931 24333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:10.979079 24333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:10.983498 24333 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46509
I20260808 22:59:10.983560 24404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46509 every 8 connection(s)
I20260808 22:59:10.983949 24333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:10.985153 24405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:10.987591 24405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2: Bootstrap starting.
I20260808 22:59:10.988207 24405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:10.988500 24405 log.cc:824] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:10.989290 24405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2: No bootstrap required, opened a new log
I20260808 22:59:10.990435 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24333
I20260808 22:59:10.990588 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:10.990774 1872 external_mini_cluster.cc:1428] Setting key c4dc2177409891a91b2883c48af80a09
I20260808 22:59:10.991185 24405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 22:59:10.991405 24405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:10.991521 24405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392d2630e7f64c6e8e9e8f097c4694f2, State: Initialized, Role: FOLLOWER
I20260808 22:59:10.991729 24405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 22:59:10.991837 24405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:10.991881 24405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:10.991976 24405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:10.993242 24405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 22:59:10.993341 24405 leader_election.cc:302] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 392d2630e7f64c6e8e9e8f097c4694f2; no voters:
I20260808 22:59:10.993542 24405 leader_election.cc:288] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:10.993587 24410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:10.993760 24410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [term 1 LEADER]: Becoming Leader. State: Replica: 392d2630e7f64c6e8e9e8f097c4694f2, State: Running, Role: LEADER
I20260808 22:59:10.993840 24405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:10.993950 24410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 22:59:10.994374 24410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 392d2630e7f64c6e8e9e8f097c4694f2. Latest consensus state: current_term: 1 leader_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260808 22:59:10.994467 24412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392d2630e7f64c6e8e9e8f097c4694f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260808 22:59:10.994576 24412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:10.994501 24410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:10.995354 24419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:10.995991 24419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:10.997819 24419 catalog_manager.cc:1347] Generated new cluster ID: 746b1dcfd7524a5ca54adabca0960220
I20260808 22:59:10.997879 24419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:10 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:59:11.003307 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:10.991423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:42483 (local address 127.0.0.1:46509)
0808 22:59:10.991597 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:10.991606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:10.991643 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:59:10.991753 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:10.991759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:10.991805 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:10.991971 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:10.992848 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:10.994010 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:10.995034 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:10.995896 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:10.996030 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:10.999101 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:10.999106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:10.999107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:10.999122 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:11.001268 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:11.001752 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:11.001757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:11.001757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:11.001912 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:11.002155 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:11.002156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:11.002156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:11.002503 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:11.002697 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:11.002784 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 22:59:11.002899 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260808 22:59:11.004468 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:11.010066 24419 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:11.010609 24419 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:11.014088 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:11.006281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:34369 (local address 127.0.0.1:46509)
0808 22:59:11.006320 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:11.006321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:11.006344 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:59:11.006368 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:11.006369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:11.006379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:11.006419 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:11.007146 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:11.007927 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:11.009029 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:11.009353 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:11.009460 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:11.010998 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:11.011000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:11.011000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:11.011007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:11.012715 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:11.013189 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:11.013190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:11.013191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:11.013262 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:11.013497 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:11.013498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:11.013498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:11.013631 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:11.013730 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:11.013891 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:59:11.013966 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:11.023918 24419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 392d2630e7f64c6e8e9e8f097c4694f2: Generated new TSK 0
I20260808 22:59:11.024431 24419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:11.028232 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24333
W20260808 22:59:11.035599 24315 connection.cc:535] client connection to 127.0.0.1:46509 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46509): Connection reset by peer (error 104)
2026-08-08T22:59:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:11 dist-test-slave-1x4g krb5kdc[24439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:11 dist-test-slave-1x4g krb5kdc[24439](info): set up 2 sockets
Aug 08 22:59:11 dist-test-slave-1x4g krb5kdc[24439](info): commencing operation
krb5kdc: starting...
W20260808 22:59:12.108923 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:12 dist-test-slave-1x4g krb5kdc[24439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:12.243862 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:12.329506 24453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:12.329784 24453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:12.329825 24453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:12.331067 24453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:12.331146 24453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:12.331176 24453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:12.331207 24453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:12.331229 24453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:12.333204 24453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:12.333544 24453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:12.333783 24453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:12.339002 24453 server_base.cc:1034] running on GCE node
W20260808 22:59:12.338959 24459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:12.338963 24460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:12.338963 24463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:12.339576 24453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:12.339861 24453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:12.341029 24453 hybrid_clock.cc:648] HybridClock initialized: now 1786229952341019 us; error 38 us; skew 500 ppm
Aug 08 22:59:12 dist-test-slave-1x4g krb5kdc[24439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:12.343587 24453 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:12.344019 24453 webserver.cc:460] Webserver started at http://127.0.0.1:38911/ using document root <none> and password file <none>
I20260808 22:59:12.344244 24453 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:12.344286 24453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:12.344389 24453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:12.345631 24453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "77110a294ae5463a8ce4b5d875579c96"
format_stamp: "Formatted at 2026-08-08 22:59:12 on dist-test-slave-1x4g"
server_key: "b12b7306d1820d2a7106b1b89884cf1e"
server_key_iv: "69c0650a8e1744e2155ee9d92bcbaff3"
server_key_version: "encryptionkey@0"
I20260808 22:59:12.346015 24453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "77110a294ae5463a8ce4b5d875579c96"
format_stamp: "Formatted at 2026-08-08 22:59:12 on dist-test-slave-1x4g"
server_key: "b12b7306d1820d2a7106b1b89884cf1e"
server_key_iv: "69c0650a8e1744e2155ee9d92bcbaff3"
server_key_version: "encryptionkey@0"
I20260808 22:59:12.347443 24453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:12.348227 24470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:12.348404 24453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:12.348497 24453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "77110a294ae5463a8ce4b5d875579c96"
format_stamp: "Formatted at 2026-08-08 22:59:12 on dist-test-slave-1x4g"
server_key: "b12b7306d1820d2a7106b1b89884cf1e"
server_key_iv: "69c0650a8e1744e2155ee9d92bcbaff3"
server_key_version: "encryptionkey@0"
I20260808 22:59:12.348562 24453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:12.358392 24453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:12.360929 24453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:12.361058 24453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:12.365411 24453 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45465
I20260808 22:59:12.365483 24526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45465 every 8 connection(s)
I20260808 22:59:12.365878 24453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:12.367034 24527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:12.369742 24527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96: Bootstrap starting.
I20260808 22:59:12.370038 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24453
I20260808 22:59:12.370170 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:12.370393 1872 external_mini_cluster.cc:1428] Setting key 9b01592cfba827005b2c9b92b2aee534
I20260808 22:59:12.370625 24527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:12.371021 24527 log.cc:824] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:12.371889 24527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96: No bootstrap required, opened a new log
I20260808 22:59:12.374079 24527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } }
I20260808 22:59:12.374334 24527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:12.374394 24527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77110a294ae5463a8ce4b5d875579c96, State: Initialized, Role: FOLLOWER
I20260808 22:59:12.374549 24527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } }
I20260808 22:59:12.374624 24527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:12.374653 24527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:12.374698 24527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:12.375378 24527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } }
I20260808 22:59:12.375476 24527 leader_election.cc:302] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77110a294ae5463a8ce4b5d875579c96; no voters:
I20260808 22:59:12.375695 24527 leader_election.cc:288] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:12.375761 24532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:12.375960 24532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [term 1 LEADER]: Becoming Leader. State: Replica: 77110a294ae5463a8ce4b5d875579c96, State: Running, Role: LEADER
I20260808 22:59:12.375965 24527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:12.376824 24532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } }
I20260808 22:59:12.377471 24534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77110a294ae5463a8ce4b5d875579c96. Latest consensus state: current_term: 1 leader_uuid: "77110a294ae5463a8ce4b5d875579c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } } }
I20260808 22:59:12.377486 24533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77110a294ae5463a8ce4b5d875579c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77110a294ae5463a8ce4b5d875579c96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45465 } } }
I20260808 22:59:12.377570 24534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:12.377677 24533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:12.377902 24544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:12.378495 24544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:12.378820 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:12.371030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:60647 (local address 127.0.0.1:45465)
0808 22:59:12.371300 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:12.371309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:12.371346 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:59:12.372027 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:12.372038 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:12.372115 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:12.372271 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:12.372512 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:12.374765 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:12.375637 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:12.376281 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:12.376378 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:12.377653 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:12.377657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:12.377658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:12.378096 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:12.378146 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:12.378165 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:12.378506 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:12.378555 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 22:59:12.378586 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
I20260808 22:59:12.379921 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:12.380970 24544 catalog_manager.cc:1347] Generated new cluster ID: 18e4549b7c10435480e3aa83ef0c3d1a
I20260808 22:59:12.381083 24544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:12.385385 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:12.381697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:58775 (local address 127.0.0.1:45465)
0808 22:59:12.381739 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:12.381741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:12.381749 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:12.381772 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:12.381773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:12.381785 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:12.381852 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:12.381998 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:12.382786 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:12.383767 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:12.384105 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:12.384188 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:12.385034 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:12.385036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:12.385037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:12.385044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:12.385067 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:12.385074 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:12.385151 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:12.385287 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 22:59:12.385302 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:59:12.400463 24544 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:12.401290 24544 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:12.408821 24544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77110a294ae5463a8ce4b5d875579c96: Generated new TSK 0
I20260808 22:59:12.409078 24544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:12.419526 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24453
W20260808 22:59:12.426209 24437 connection.cc:535] client connection to 127.0.0.1:45465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45465): Connection reset by peer (error 104)
2026-08-08T22:59:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:12 dist-test-slave-1x4g krb5kdc[24560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:12 dist-test-slave-1x4g krb5kdc[24560](info): set up 2 sockets
Aug 08 22:59:12 dist-test-slave-1x4g krb5kdc[24560](info): commencing operation
krb5kdc: starting...
W20260808 22:59:13.508971 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:13 dist-test-slave-1x4g krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:13.648154 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:13.741358 24573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:13.741657 24573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:13.741704 24573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:13.742980 24573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:13.743068 24573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:13.743098 24573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:13.743129 24573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:13.743163 24573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:13.744946 24573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:13.745278 24573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:13.745553 24573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:13.751219 24581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:13.751221 24578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:13.751219 24579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:13.751607 24573 server_base.cc:1034] running on GCE node
I20260808 22:59:13.751899 24573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:13.752249 24573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:13.753455 24573 hybrid_clock.cc:648] HybridClock initialized: now 1786229953753445 us; error 56 us; skew 500 ppm
Aug 08 22:59:13 dist-test-slave-1x4g krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:13.760877 24573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:13.761471 24573 webserver.cc:460] Webserver started at http://127.0.0.1:45237/ using document root <none> and password file <none>
I20260808 22:59:13.761740 24573 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:13.761793 24573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:13.761942 24573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:13.763144 24573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "dbf490f7e52b4bc694ae64ef3857e884"
format_stamp: "Formatted at 2026-08-08 22:59:13 on dist-test-slave-1x4g"
server_key: "8af2e0162b94068e711282d00022d061"
server_key_iv: "6f00858da2e0b5b6b6af9831a4e21fc9"
server_key_version: "encryptionkey@0"
I20260808 22:59:13.763545 24573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "dbf490f7e52b4bc694ae64ef3857e884"
format_stamp: "Formatted at 2026-08-08 22:59:13 on dist-test-slave-1x4g"
server_key: "8af2e0162b94068e711282d00022d061"
server_key_iv: "6f00858da2e0b5b6b6af9831a4e21fc9"
server_key_version: "encryptionkey@0"
I20260808 22:59:13.765266 24573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:13.766220 24588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:13.766484 24573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:13.766606 24573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "dbf490f7e52b4bc694ae64ef3857e884"
format_stamp: "Formatted at 2026-08-08 22:59:13 on dist-test-slave-1x4g"
server_key: "8af2e0162b94068e711282d00022d061"
server_key_iv: "6f00858da2e0b5b6b6af9831a4e21fc9"
server_key_version: "encryptionkey@0"
I20260808 22:59:13.766710 24573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:13.770606 24573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:13.773176 24573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:13.773368 24573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:13.778487 24573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37999
I20260808 22:59:13.778578 24646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37999 every 8 connection(s)
I20260808 22:59:13.778985 24573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:13.780102 24647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:13.782773 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884: Bootstrap starting.
I20260808 22:59:13.783392 24647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:13.783679 24647 log.cc:824] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:13.784538 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884: No bootstrap required, opened a new log
I20260808 22:59:13.785588 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24573
I20260808 22:59:13.785732 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:13.785926 1872 external_mini_cluster.cc:1428] Setting key a0d8ca3c01be2ca45b38a8fa2a08fa4b
I20260808 22:59:13.786685 24647 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260808 22:59:13.786900 24647 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:13.786950 24647 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf490f7e52b4bc694ae64ef3857e884, State: Initialized, Role: FOLLOWER
I20260808 22:59:13.787106 24647 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260808 22:59:13.787207 24647 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:13.787249 24647 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:13.787357 24647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:13.788424 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:13.786587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:39415 (local address 127.0.0.1:37999)
0808 22:59:13.786782 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:13.786791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:13.786821 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:59:13.786953 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:13.786962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:13.786996 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:13.787168 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:13.787896 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:13.787899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:13.787899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:13.787903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:13.787951 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:13.787962 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:13.788020 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:13.788124 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 22:59:13.788162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260808 22:59:13.788815 24647 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260808 22:59:13.788949 24647 leader_election.cc:302] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf490f7e52b4bc694ae64ef3857e884; no voters:
I20260808 22:59:13.789155 24647 leader_election.cc:288] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:13.789191 24652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:13.789353 24652 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [term 1 LEADER]: Becoming Leader. State: Replica: dbf490f7e52b4bc694ae64ef3857e884, State: Running, Role: LEADER
I20260808 22:59:13.789463 24647 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:13.789496 24652 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } }
I20260808 22:59:13.789549 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:13.790045 24652 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf490f7e52b4bc694ae64ef3857e884. Latest consensus state: current_term: 1 leader_uuid: "dbf490f7e52b4bc694ae64ef3857e884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } } }
I20260808 22:59:13.790170 24653 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf490f7e52b4bc694ae64ef3857e884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf490f7e52b4bc694ae64ef3857e884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37999 } } }
I20260808 22:59:13.790271 24653 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:13.790237 24652 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:13.790735 24661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:13.791414 24661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:13.793324 24661 catalog_manager.cc:1347] Generated new cluster ID: 77797be847d14c89930f77cfac03b7ca
I20260808 22:59:13.793392 24661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:13.798256 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:13.791311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40339 (local address 127.0.0.1:37999)
0808 22:59:13.791350 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:13.791353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:13.797042 (+ 5689us) server_negotiation.cc:407] Connection header received
0808 22:59:13.797126 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:13.797127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:13.797134 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:13.797187 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:13.797907 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:13.797909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:13.797909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:13.797914 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:13.797933 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:13.797939 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:13.798017 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:13.798068 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 22:59:13.798086 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:59:13.800989 24661 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:13.801195 24661 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:13.805550 24661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf490f7e52b4bc694ae64ef3857e884: Generated new TSK 0
I20260808 22:59:13.805770 24661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:13.809681 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24573
2026-08-08T22:59:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:13 dist-test-slave-1x4g krb5kdc[24678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:13 dist-test-slave-1x4g krb5kdc[24678](info): set up 2 sockets
Aug 08 22:59:13 dist-test-slave-1x4g krb5kdc[24678](info): commencing operation
krb5kdc: starting...
W20260808 22:59:14.905289 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:14 dist-test-slave-1x4g krb5kdc[24678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:15.042007 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:15.129522 24691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:15.129833 24691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:15.129885 24691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:15.131505 24691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:15.131587 24691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:15.131614 24691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:15.131647 24691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:15.131680 24691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:15.133510 24691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:15.133849 24691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:15.134135 24691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:15.139948 24691 server_base.cc:1034] running on GCE node
W20260808 22:59:15.139894 24697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:15.139875 24701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:15.139873 24699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:15.140491 24691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:15.140908 24691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:15.142081 24691 hybrid_clock.cc:648] HybridClock initialized: now 1786229955142068 us; error 39 us; skew 500 ppm
Aug 08 22:59:15 dist-test-slave-1x4g krb5kdc[24678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:15.145113 24691 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:15.145685 24691 webserver.cc:460] Webserver started at http://127.0.0.1:36823/ using document root <none> and password file <none>
I20260808 22:59:15.145965 24691 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:15.146045 24691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:15.146219 24691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:15.147455 24691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "241f05c3283e40d6954dbe3ca2ad86a2"
format_stamp: "Formatted at 2026-08-08 22:59:15 on dist-test-slave-1x4g"
server_key: "15234d1dbe5b3af15e0d29aa0acb1749"
server_key_iv: "ce855d9b172a67393cf57ab3ac350140"
server_key_version: "encryptionkey@0"
I20260808 22:59:15.147877 24691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "241f05c3283e40d6954dbe3ca2ad86a2"
format_stamp: "Formatted at 2026-08-08 22:59:15 on dist-test-slave-1x4g"
server_key: "15234d1dbe5b3af15e0d29aa0acb1749"
server_key_iv: "ce855d9b172a67393cf57ab3ac350140"
server_key_version: "encryptionkey@0"
I20260808 22:59:15.149562 24691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 22:59:15.150421 24708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:15.150607 24691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:15.150732 24691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "241f05c3283e40d6954dbe3ca2ad86a2"
format_stamp: "Formatted at 2026-08-08 22:59:15 on dist-test-slave-1x4g"
server_key: "15234d1dbe5b3af15e0d29aa0acb1749"
server_key_iv: "ce855d9b172a67393cf57ab3ac350140"
server_key_version: "encryptionkey@0"
I20260808 22:59:15.150833 24691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:15.164687 24691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:15.167454 24691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:15.167646 24691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:15.172519 24691 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36551
I20260808 22:59:15.172567 24762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36551 every 8 connection(s)
I20260808 22:59:15.173012 24691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:15.174088 24763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:15.176630 24763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2: Bootstrap starting.
I20260808 22:59:15.177342 24763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:15.177683 24763 log.cc:824] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:15.178582 24763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2: No bootstrap required, opened a new log
I20260808 22:59:15.178694 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24691
I20260808 22:59:15.178814 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:15.179006 1872 external_mini_cluster.cc:1428] Setting key 3f096737947110db7427038020e13d63
I20260808 22:59:15.180750 24763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } }
I20260808 22:59:15.181008 24763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:15.181054 24763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241f05c3283e40d6954dbe3ca2ad86a2, State: Initialized, Role: FOLLOWER
I20260808 22:59:15.181209 24763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } }
I20260808 22:59:15.181303 24763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:15.181337 24763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:15.181402 24763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:15.182494 24763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } }
I20260808 22:59:15.182590 24763 leader_election.cc:302] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 241f05c3283e40d6954dbe3ca2ad86a2; no voters:
I20260808 22:59:15.182801 24763 leader_election.cc:288] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:15.182822 24770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:15.182948 24770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [term 1 LEADER]: Becoming Leader. State: Replica: 241f05c3283e40d6954dbe3ca2ad86a2, State: Running, Role: LEADER
I20260808 22:59:15.183084 24770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } }
I20260808 22:59:15.183089 24763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:15.183531 24772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 241f05c3283e40d6954dbe3ca2ad86a2. Latest consensus state: current_term: 1 leader_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } } }
I20260808 22:59:15.183763 24772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:15.183691 24771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241f05c3283e40d6954dbe3ca2ad86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36551 } } }
I20260808 22:59:15.184217 24771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:15.184391 24779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:15.184942 24779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:15.186713 24779 catalog_manager.cc:1347] Generated new cluster ID: 3db6c73f417046b38da4ca20246ee464
I20260808 22:59:15.186780 24779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:15 dist-test-slave-1x4g krb5kdc[24678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:59:15.192045 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:15.179599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:60597 (local address 127.0.0.1:36551)
0808 22:59:15.179830 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:15.179838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:15.179866 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:59:15.180724 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:15.180731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:15.180838 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:15.180972 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:15.181818 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:15.182885 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:15.184018 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:15.184742 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:15.184889 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:15.187829 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:15.187834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:15.187834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:15.187849 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:15.190009 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:15.190528 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:15.190532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:15.190532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:15.190677 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:15.190930 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:15.190931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:15.190932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:15.191292 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:15.191466 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:15.191619 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 22:59:15.191765 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260808 22:59:15.193269 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:15.203017 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:15.194769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:51089 (local address 127.0.0.1:36551)
0808 22:59:15.194811 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:15.194813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:15.195085 (+ 272us) server_negotiation.cc:407] Connection header received
0808 22:59:15.195191 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:15.195192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:15.195203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:15.195233 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:15.196008 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:15.196871 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:15.197904 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:15.198262 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:15.198362 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:15.199859 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:15.199861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:15.199861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:15.199870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:15.201622 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:15.202106 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:15.202108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:15.202108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:15.202181 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:15.202419 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:15.202420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:15.202420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:15.202569 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:15.202680 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:15.202834 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 22:59:15.202910 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:15.212097 24779 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:15.212944 24779 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:15.227469 24779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 241f05c3283e40d6954dbe3ca2ad86a2: Generated new TSK 0
I20260808 22:59:15.227734 24779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:15.239971 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24691
W20260808 22:59:15.245995 24676 connection.cc:535] client connection to 127.0.0.1:36551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36551): Connection reset by peer (error 104)
2026-08-08T22:59:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:15 dist-test-slave-1x4g krb5kdc[24798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:15 dist-test-slave-1x4g krb5kdc[24798](info): set up 2 sockets
Aug 08 22:59:15 dist-test-slave-1x4g krb5kdc[24798](info): commencing operation
krb5kdc: starting...
W20260808 22:59:16.313056 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:16 dist-test-slave-1x4g krb5kdc[24798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:16.452983 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:16.540150 24811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:16.540413 24811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:16.540457 24811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:16.542518 24811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:16.542613 24811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:16.542639 24811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:16.542660 24811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:16.542694 24811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:16.544535 24811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:16.544934 24811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:16.545223 24811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:16.550992 24816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:16.550978 24817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:16.551038 24811 server_base.cc:1034] running on GCE node
W20260808 22:59:16.550969 24819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:16.551543 24811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:16.551890 24811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:16.553043 24811 hybrid_clock.cc:648] HybridClock initialized: now 1786229956553039 us; error 43 us; skew 500 ppm
Aug 08 22:59:16 dist-test-slave-1x4g krb5kdc[24798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229956, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:16.555842 24811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:16.556391 24811 webserver.cc:460] Webserver started at http://127.0.0.1:44119/ using document root <none> and password file <none>
I20260808 22:59:16.556636 24811 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:16.556682 24811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:16.556846 24811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:16.557952 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "06f064664a3f4a479811eb7dc04ebac7"
format_stamp: "Formatted at 2026-08-08 22:59:16 on dist-test-slave-1x4g"
server_key: "40b88bf4f0c29bd7842e89e8761c04ae"
server_key_iv: "c7589afecf9404c14ea314476369d818"
server_key_version: "encryptionkey@0"
I20260808 22:59:16.558283 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "06f064664a3f4a479811eb7dc04ebac7"
format_stamp: "Formatted at 2026-08-08 22:59:16 on dist-test-slave-1x4g"
server_key: "40b88bf4f0c29bd7842e89e8761c04ae"
server_key_iv: "c7589afecf9404c14ea314476369d818"
server_key_version: "encryptionkey@0"
I20260808 22:59:16.559751 24811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:16.560578 24826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:16.560889 24811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:16.561064 24811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "06f064664a3f4a479811eb7dc04ebac7"
format_stamp: "Formatted at 2026-08-08 22:59:16 on dist-test-slave-1x4g"
server_key: "40b88bf4f0c29bd7842e89e8761c04ae"
server_key_iv: "c7589afecf9404c14ea314476369d818"
server_key_version: "encryptionkey@0"
I20260808 22:59:16.561158 24811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:16.575417 24811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:16.578076 24811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:16.578209 24811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:16.582683 24811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36475
I20260808 22:59:16.582720 24880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36475 every 8 connection(s)
I20260808 22:59:16.583132 24811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:16.584084 24881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:16.586714 24881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7: Bootstrap starting.
I20260808 22:59:16.587375 24881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:16.587678 24881 log.cc:824] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:16.588491 24881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7: No bootstrap required, opened a new log
I20260808 22:59:16.589267 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24811
I20260808 22:59:16.589403 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:16.589596 1872 external_mini_cluster.cc:1428] Setting key 6a92a1dedae8b1fdae04a3c25c362e84
I20260808 22:59:16.590656 24881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } }
I20260808 22:59:16.590829 24881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:16.590898 24881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f064664a3f4a479811eb7dc04ebac7, State: Initialized, Role: FOLLOWER
I20260808 22:59:16.591064 24881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } }
I20260808 22:59:16.591157 24881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:16.591202 24881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:16.591255 24881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:16.591907 24881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } }
I20260808 22:59:16.592037 24881 leader_election.cc:302] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06f064664a3f4a479811eb7dc04ebac7; no voters:
I20260808 22:59:16.592249 24881 leader_election.cc:288] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:16.592281 24887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:16.592404 24887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [term 1 LEADER]: Becoming Leader. State: Replica: 06f064664a3f4a479811eb7dc04ebac7, State: Running, Role: LEADER
I20260808 22:59:16.592566 24881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:16.592554 24887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } }
I20260808 22:59:16.594007 24889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06f064664a3f4a479811eb7dc04ebac7. Latest consensus state: current_term: 1 leader_uuid: "06f064664a3f4a479811eb7dc04ebac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } } }
I20260808 22:59:16.594151 24889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:16.594398 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06f064664a3f4a479811eb7dc04ebac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f064664a3f4a479811eb7dc04ebac7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36475 } } }
I20260808 22:59:16.594409 24903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:16.594502 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:16.595454 24903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:16.596082 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:16.590209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:36887 (local address 127.0.0.1:36475)
0808 22:59:16.590425 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:16.590435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:16.590463 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:59:16.590610 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:16.590616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:16.590666 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:16.590811 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:16.591009 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:16.592597 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:16.593767 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:16.594432 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:16.594558 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:16.595340 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:16.595345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:16.595345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:16.595604 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:16.595651 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:16.595664 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:16.595768 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:16.595854 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 22:59:16.595893 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260808 22:59:16.597230 24903 catalog_manager.cc:1347] Generated new cluster ID: 2cd7d8e6bd1745e8bcfa7a8b83a3a95f
I20260808 22:59:16.597285 24903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:16.597313 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:16.602231 24903 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:16.602932 24903 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:16.603129 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:16.598985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:52057 (local address 127.0.0.1:36475)
0808 22:59:16.599029 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:16.599040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:16.599191 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:59:16.599258 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:16.599259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:16.599271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:16.599320 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:16.599475 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:16.600224 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:16.601607 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:16.601876 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:16.601945 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:16.602823 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:16.602824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:16.602824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:16.602830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:16.602847 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:16.602852 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:16.602886 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:16.603023 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 22:59:16.603050 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:16.607429 24903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06f064664a3f4a479811eb7dc04ebac7: Generated new TSK 0
I20260808 22:59:16.607671 24903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:16.612996 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24811
W20260808 22:59:16.621436 24796 connection.cc:535] client connection to 127.0.0.1:36475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36475): Connection reset by peer (error 104)
2026-08-08T22:59:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:16 dist-test-slave-1x4g krb5kdc[24914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:16 dist-test-slave-1x4g krb5kdc[24914](info): set up 2 sockets
Aug 08 22:59:16 dist-test-slave-1x4g krb5kdc[24914](info): commencing operation
krb5kdc: starting...
W20260808 22:59:17.721187 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:17 dist-test-slave-1x4g krb5kdc[24914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:59:17.861537 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:17.953842 24929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:17.954136 24929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:17.954191 24929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:17.955488 24929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:17.955565 24929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:17.955591 24929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:17.955612 24929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:17.955644 24929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:17.957551 24929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:17.957878 24929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:17.958173 24929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:17.964493 24929 server_base.cc:1034] running on GCE node
W20260808 22:59:17.964543 24934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:17.964509 24935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:17.964721 24937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:17.965075 24929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:17.965428 24929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:17.966614 24929 hybrid_clock.cc:648] HybridClock initialized: now 1786229957966598 us; error 34 us; skew 500 ppm
Aug 08 22:59:17 dist-test-slave-1x4g krb5kdc[24914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:17.969633 24929 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:59:17.970168 24929 webserver.cc:460] Webserver started at http://127.0.0.1:46559/ using document root <none> and password file <none>
I20260808 22:59:17.970443 24929 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:17.970517 24929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:17.970666 24929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:17.971813 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "1e5249c8628b4539a8d3070d7a665e37"
format_stamp: "Formatted at 2026-08-08 22:59:17 on dist-test-slave-1x4g"
server_key: "87e3d09369ae10a80a973333686c4825"
server_key_iv: "b21c42c1e98ec4756b74ee9b1d6a5929"
server_key_version: "encryptionkey@0"
I20260808 22:59:17.972200 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "1e5249c8628b4539a8d3070d7a665e37"
format_stamp: "Formatted at 2026-08-08 22:59:17 on dist-test-slave-1x4g"
server_key: "87e3d09369ae10a80a973333686c4825"
server_key_iv: "b21c42c1e98ec4756b74ee9b1d6a5929"
server_key_version: "encryptionkey@0"
I20260808 22:59:17.973817 24929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:17.974710 24944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:17.975101 24929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:17.975334 24929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "1e5249c8628b4539a8d3070d7a665e37"
format_stamp: "Formatted at 2026-08-08 22:59:17 on dist-test-slave-1x4g"
server_key: "87e3d09369ae10a80a973333686c4825"
server_key_iv: "b21c42c1e98ec4756b74ee9b1d6a5929"
server_key_version: "encryptionkey@0"
I20260808 22:59:17.975471 24929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:17.979597 24929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:17.982175 24929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:17.982362 24929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:17.986887 24929 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39185
I20260808 22:59:17.986958 25000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39185 every 8 connection(s)
I20260808 22:59:17.987349 24929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:17.988328 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 24929
I20260808 22:59:17.988453 25001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:17.988484 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:17.988695 1872 external_mini_cluster.cc:1428] Setting key adc9fab943843a8220bd19194246620f
I20260808 22:59:17.991468 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:17.989340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:55883 (local address 127.0.0.1:39185)
0808 22:59:17.989558 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:17.989566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:17.989594 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:59:17.990072 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:17.990080 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:17.990106 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:17.990269 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:17.990998 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:17.991002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:17.991002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:17.991006 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:17.991060 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:17.991073 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:17.991154 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:17.991226 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:59:17.991274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260808 22:59:17.992033 25001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37: Bootstrap starting.
I20260808 22:59:17.992601 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:17.992921 25001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:17.993263 25001 log.cc:824] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:17.994078 25001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37: No bootstrap required, opened a new log
I20260808 22:59:17.995724 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:17.994374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:38035 (local address 127.0.0.1:39185)
0808 22:59:17.994421 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:17.994423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:17.994544 (+ 121us) server_negotiation.cc:407] Connection header received
0808 22:59:17.994620 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:17.994621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:17.994626 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:17.994670 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:17.995365 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:17.995367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:17.995367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:17.995371 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:17.995390 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:17.995395 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:17.995448 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:17.995529 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 22:59:17.995541 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:17.996858 25001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } }
I20260808 22:59:17.997043 25001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:17.997102 25001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5249c8628b4539a8d3070d7a665e37, State: Initialized, Role: FOLLOWER
I20260808 22:59:17.997236 25001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } }
I20260808 22:59:17.997330 25001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:17.997377 25001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:17.997429 25001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:17.998204 25001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } }
I20260808 22:59:17.998330 25001 leader_election.cc:302] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e5249c8628b4539a8d3070d7a665e37; no voters:
I20260808 22:59:17.998539 25001 leader_election.cc:288] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:17.998638 25014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:17.998805 25014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [term 1 LEADER]: Becoming Leader. State: Replica: 1e5249c8628b4539a8d3070d7a665e37, State: Running, Role: LEADER
I20260808 22:59:17.998812 25001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:17.998914 25014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } }
I20260808 22:59:17.999646 25015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e5249c8628b4539a8d3070d7a665e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } } }
I20260808 22:59:17.999773 25015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:17.999944 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e5249c8628b4539a8d3070d7a665e37. Latest consensus state: current_term: 1 leader_uuid: "1e5249c8628b4539a8d3070d7a665e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e5249c8628b4539a8d3070d7a665e37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39185 } } }
I20260808 22:59:18.000005 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:18.000005 25023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:18.000623 25023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:18.002578 25023 catalog_manager.cc:1347] Generated new cluster ID: 7e5ba0472ce84cc2b4dfd92a6405b2a2
I20260808 22:59:18.002647 25023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:18.013648 25023 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:18.013865 25023 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:18.022186 25023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e5249c8628b4539a8d3070d7a665e37: Generated new TSK 0
I20260808 22:59:18.022432 25023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:18.029417 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 24929
2026-08-08T22:59:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:18 dist-test-slave-1x4g krb5kdc[25035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:18 dist-test-slave-1x4g krb5kdc[25035](info): set up 2 sockets
Aug 08 22:59:18 dist-test-slave-1x4g krb5kdc[25035](info): commencing operation
krb5kdc: starting...
W20260808 22:59:19.116343 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:19.253063 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36931
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:41305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:36931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:19.339638 25051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:19.339881 25051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:19.339907 25051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:19.341226 25051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:19.341284 25051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:19.341300 25051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:19.341313 25051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:19.341325 25051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:19.343120 25051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:41305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:36931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:36931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:19.343421 25051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:19.343699 25051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:19.349110 25059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:19.349123 25057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:19.349345 25051 server_base.cc:1034] running on GCE node
W20260808 22:59:19.349135 25056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:19.349675 25051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:19.350010 25051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:19.351197 25051 hybrid_clock.cc:648] HybridClock initialized: now 1786229959351195 us; error 38 us; skew 500 ppm
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229959, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:19.359062 25051 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:19.359659 25051 webserver.cc:460] Webserver started at http://127.1.212.62:35709/ using document root <none> and password file <none>
I20260808 22:59:19.359915 25051 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:19.359992 25051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:19.360162 25051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:19.361418 25051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "bc263a3fc31440a19d1401511887ab89"
format_stamp: "Formatted at 2026-08-08 22:59:19 on dist-test-slave-1x4g"
server_key: "1831e8b39b634075bdf66c89a949e8bc"
server_key_iv: "4e19af0ec160cf7f4a0b9fd96c803eb5"
server_key_version: "encryptionkey@0"
I20260808 22:59:19.361915 25051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "bc263a3fc31440a19d1401511887ab89"
format_stamp: "Formatted at 2026-08-08 22:59:19 on dist-test-slave-1x4g"
server_key: "1831e8b39b634075bdf66c89a949e8bc"
server_key_iv: "4e19af0ec160cf7f4a0b9fd96c803eb5"
server_key_version: "encryptionkey@0"
I20260808 22:59:19.363451 25051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:19.364346 25067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:19.364562 25051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:19.364684 25051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "bc263a3fc31440a19d1401511887ab89"
format_stamp: "Formatted at 2026-08-08 22:59:19 on dist-test-slave-1x4g"
server_key: "1831e8b39b634075bdf66c89a949e8bc"
server_key_iv: "4e19af0ec160cf7f4a0b9fd96c803eb5"
server_key_version: "encryptionkey@0"
I20260808 22:59:19.364770 25051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:19.386070 25051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:19.388618 25051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:19.388855 25051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:19.393613 25051 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:36931
I20260808 22:59:19.393657 25125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:36931 every 8 connection(s)
I20260808 22:59:19.394088 25051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:19.395110 25126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:19.397724 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89: Bootstrap starting.
I20260808 22:59:19.398346 25126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:19.398619 25126 log.cc:824] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:19.399372 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89: No bootstrap required, opened a new log
I20260808 22:59:19.399765 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25051
I20260808 22:59:19.399869 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:19.400063 1872 external_mini_cluster.cc:1428] Setting key 321bc299b1496a5f97dc46a38363c296
I20260808 22:59:19.401726 25126 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } }
I20260808 22:59:19.401921 25126 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:19.401960 25126 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc263a3fc31440a19d1401511887ab89, State: Initialized, Role: FOLLOWER
I20260808 22:59:19.402149 25126 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } }
I20260808 22:59:19.402253 25126 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:19.402288 25126 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:19.402347 25126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:19.403501 25126 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } }
I20260808 22:59:19.403607 25126 leader_election.cc:302] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc263a3fc31440a19d1401511887ab89; no voters:
I20260808 22:59:19.403836 25126 leader_election.cc:288] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:19.403867 25132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:19.403998 25132 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [term 1 LEADER]: Becoming Leader. State: Replica: bc263a3fc31440a19d1401511887ab89, State: Running, Role: LEADER
I20260808 22:59:19.404148 25132 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } }
I20260808 22:59:19.404152 25126 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:19.404609 25133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc263a3fc31440a19d1401511887ab89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } } }
I20260808 22:59:19.404692 25133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:19.405009 25134 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc263a3fc31440a19d1401511887ab89. Latest consensus state: current_term: 1 leader_uuid: "bc263a3fc31440a19d1401511887ab89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc263a3fc31440a19d1401511887ab89" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 36931 } } }
I20260808 22:59:19.405153 25134 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:19.405367 25137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:19.406000 25137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:19.408051 25137 catalog_manager.cc:1347] Generated new cluster ID: 8b27292fa67b40788c6ca41e117fd15a
I20260808 22:59:19.408113 25137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:19.412367 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:19.400679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39533 (local address 127.1.212.62:36931)
0808 22:59:19.400934 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:19.400943 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:19.400971 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:59:19.401692 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:19.401702 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:19.401762 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:19.401895 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:19.402779 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:19.403857 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:19.405026 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:19.405681 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:19.406195 (+ 514us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:19.408804 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:19.408810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:19.408810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:19.408826 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:19.410641 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:19.411078 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:19.411082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:19.411082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:19.411210 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:19.411459 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:19.411460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:19.411461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:19.411751 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:19.411992 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:19.412085 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:59:19.412197 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":145,"threads_started":1}
I20260808 22:59:19.413576 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:19.428128 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:19.421240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47903 (local address 127.1.212.62:36931)
0808 22:59:19.421298 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:19.421300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:19.421464 (+ 164us) server_negotiation.cc:407] Connection header received
0808 22:59:19.421556 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:19.421557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:19.421569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:19.421643 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:19.422274 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:19.422803 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:19.423566 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:19.423797 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:19.423881 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:19.425310 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:19.425312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:19.425313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:19.425322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:19.426848 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:19.427257 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:19.427259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:19.427259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:19.427321 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:19.427571 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:19.427572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:19.427573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:19.427695 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:19.427826 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:19.427938 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 22:59:19.428009 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:59:19.429625 25137 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:19.430143 25137 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:19.435160 25137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc263a3fc31440a19d1401511887ab89: Generated new TSK 0
I20260808 22:59:19.435386 25137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:19.440222 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25051
W20260808 22:59:19.448225 25033 connection.cc:535] client connection to 127.1.212.62:36931 recv error: Network error: failed to read from TLS socket (remote: 127.1.212.62:36931): Connection reset by peer (error 104)
2026-08-08T22:59:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25159](info): set up 2 sockets
Aug 08 22:59:19 dist-test-slave-1x4g krb5kdc[25159](info): commencing operation
krb5kdc: starting...
W20260808 22:59:20.530405 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:20 dist-test-slave-1x4g krb5kdc[25159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:20.663957 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39655
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:39655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:20.748708 25173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:20.749004 25173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:20.749045 25173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:20.750267 25173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:20.750348 25173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:20.750375 25173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:20.750415 25173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:20.750439 25173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:20.752022 25173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:39655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:20.752329 25173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:20.752594 25173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:20.757823 25179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:20.757823 25180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:20.757885 25173 server_base.cc:1034] running on GCE node
W20260808 22:59:20.757823 25182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:20.758394 25173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:20.758715 25173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:20.759888 25173 hybrid_clock.cc:648] HybridClock initialized: now 1786229960759875 us; error 39 us; skew 500 ppm
Aug 08 22:59:20 dist-test-slave-1x4g krb5kdc[25159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229960, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:20.762650 25173 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:20.763111 25173 webserver.cc:460] Webserver started at http://127.1.212.62:39579/ using document root <none> and password file <none>
I20260808 22:59:20.763370 25173 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:20.763439 25173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:20.763594 25173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:20.764711 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "fd9eb689034345939018dbb172344cbb"
format_stamp: "Formatted at 2026-08-08 22:59:20 on dist-test-slave-1x4g"
server_key: "fad80808201afd4851fa621ac86bd4b2"
server_key_iv: "331ef6a558d6bbb4aa27cad84de402b8"
server_key_version: "encryptionkey@0"
I20260808 22:59:20.765138 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "fd9eb689034345939018dbb172344cbb"
format_stamp: "Formatted at 2026-08-08 22:59:20 on dist-test-slave-1x4g"
server_key: "fad80808201afd4851fa621ac86bd4b2"
server_key_iv: "331ef6a558d6bbb4aa27cad84de402b8"
server_key_version: "encryptionkey@0"
I20260808 22:59:20.766556 25173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:20.767477 25189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:20.767674 25173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:20.767778 25173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "fd9eb689034345939018dbb172344cbb"
format_stamp: "Formatted at 2026-08-08 22:59:20 on dist-test-slave-1x4g"
server_key: "fad80808201afd4851fa621ac86bd4b2"
server_key_iv: "331ef6a558d6bbb4aa27cad84de402b8"
server_key_version: "encryptionkey@0"
I20260808 22:59:20.767858 25173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:20.783124 25173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:20.785650 25173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:20.785844 25173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:20.790014 25173 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:39655
I20260808 22:59:20.790064 25243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:39655 every 8 connection(s)
I20260808 22:59:20.790463 25173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:20.791385 25244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:20.793932 25244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: Bootstrap starting.
I20260808 22:59:20.794564 25244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:20.794886 25244 log.cc:824] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:20.795637 25244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: No bootstrap required, opened a new log
I20260808 22:59:20.797415 25244 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } }
I20260808 22:59:20.797574 25244 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:20.797621 25244 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9eb689034345939018dbb172344cbb, State: Initialized, Role: FOLLOWER
I20260808 22:59:20.797809 25244 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } }
I20260808 22:59:20.797904 25244 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:20.797958 25244 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:20.798012 25244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:20.798967 25244 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } }
I20260808 22:59:20.799088 25244 leader_election.cc:302] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd9eb689034345939018dbb172344cbb; no voters:
I20260808 22:59:20.799288 25244 leader_election.cc:288] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:20.799360 25247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:20.799556 25247 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [term 1 LEADER]: Becoming Leader. State: Replica: fd9eb689034345939018dbb172344cbb, State: Running, Role: LEADER
I20260808 22:59:20.799576 25244 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:20.799659 25247 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } }
I20260808 22:59:20.800156 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25173
I20260808 22:59:20.800299 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:20.800499 1872 external_mini_cluster.cc:1428] Setting key d0f222220a30d7627bd04830e241fe98
W20260808 22:59:20.801602 25265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 22:59:20.801623 25249 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd9eb689034345939018dbb172344cbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } } }
W20260808 22:59:20.801735 25265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:20.801756 25249 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:20.801625 25250 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd9eb689034345939018dbb172344cbb. Latest consensus state: current_term: 1 leader_uuid: "fd9eb689034345939018dbb172344cbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9eb689034345939018dbb172344cbb" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 39655 } } }
I20260808 22:59:20.802103 25250 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:20.802103 25269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:20.802397 25269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:20.804042 25269 catalog_manager.cc:1347] Generated new cluster ID: ddc4e7d955ff42ef98a8c7234df38998
I20260808 22:59:20.804097 25269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:20.806897 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:20.801065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53855 (local address 127.1.212.62:39655)
0808 22:59:20.801299 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:20.801310 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:20.801351 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:59:20.801490 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:20.801498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:20.801559 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:20.801971 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:20.802089 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:20.803395 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:20.804523 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:20.805177 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:20.805295 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:20.806145 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:20.806150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:20.806150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:20.806374 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:20.806421 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:20.806436 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:20.806525 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:20.806644 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 22:59:20.806674 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
I20260808 22:59:20.810034 25269 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:20.810057 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:20.810750 25269 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:20.816954 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:20.813101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58061 (local address 127.1.212.62:39655)
0808 22:59:20.813145 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:20.813147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:20.813217 (+ 70us) server_negotiation.cc:407] Connection header received
0808 22:59:20.813299 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:20.813300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:20.813322 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:20.813381 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:20.813561 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:20.814315 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:20.815448 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:20.815791 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:20.815851 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:20.816650 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:20.816652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:20.816652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:20.816658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:20.816673 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:20.816678 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:20.816734 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:20.816854 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 22:59:20.816867 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:59:20.818065 25269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd9eb689034345939018dbb172344cbb: Generated new TSK 0
I20260808 22:59:20.818284 25269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:20.820617 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25173
W20260808 22:59:20.826917 25157 connection.cc:535] client connection to 127.1.212.62:39655 recv error: Network error: failed to read from TLS socket (remote: 127.1.212.62:39655): Connection reset by peer (error 104)
2026-08-08T22:59:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:20 dist-test-slave-1x4g krb5kdc[25279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:20 dist-test-slave-1x4g krb5kdc[25279](info): set up 2 sockets
Aug 08 22:59:20 dist-test-slave-1x4g krb5kdc[25279](info): commencing operation
krb5kdc: starting...
W20260808 22:59:21.917973 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:22 dist-test-slave-1x4g krb5kdc[25279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:22.061378 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46281
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:38391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:46281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:22.146799 25292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:22.147037 25292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:22.147064 25292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:22.148236 25292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:22.148283 25292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:22.148298 25292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:22.148310 25292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:22.148322 25292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:22.150121 25292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:38391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:46281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:46281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:22.150368 25292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:22.150574 25292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:22.156445 25301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:22.156452 25297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:22.156476 25299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:22.156539 25292 server_base.cc:1034] running on GCE node
I20260808 22:59:22.157029 25292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:22.157361 25292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:22.158613 25292 hybrid_clock.cc:648] HybridClock initialized: now 1786229962158596 us; error 29 us; skew 500 ppm
Aug 08 22:59:22 dist-test-slave-1x4g krb5kdc[25279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:22.161496 25292 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:22.161993 25292 webserver.cc:460] Webserver started at http://127.1.212.62:33525/ using document root <none> and password file <none>
I20260808 22:59:22.162263 25292 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:22.162312 25292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:22.162492 25292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:22.163625 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "57576cc8ecfd4219bfc70d8584f00d05"
format_stamp: "Formatted at 2026-08-08 22:59:22 on dist-test-slave-1x4g"
server_key: "6b76b27fd8525ed595c2545746e0a924"
server_key_iv: "cf583b15d8b14de17a4e97e7fd70ccf8"
server_key_version: "encryptionkey@0"
I20260808 22:59:22.164009 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "57576cc8ecfd4219bfc70d8584f00d05"
format_stamp: "Formatted at 2026-08-08 22:59:22 on dist-test-slave-1x4g"
server_key: "6b76b27fd8525ed595c2545746e0a924"
server_key_iv: "cf583b15d8b14de17a4e97e7fd70ccf8"
server_key_version: "encryptionkey@0"
I20260808 22:59:22.165607 25292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:22.166465 25308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:22.166679 25292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:22.166796 25292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "57576cc8ecfd4219bfc70d8584f00d05"
format_stamp: "Formatted at 2026-08-08 22:59:22 on dist-test-slave-1x4g"
server_key: "6b76b27fd8525ed595c2545746e0a924"
server_key_iv: "cf583b15d8b14de17a4e97e7fd70ccf8"
server_key_version: "encryptionkey@0"
I20260808 22:59:22.166877 25292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:22.170557 25292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:22.172849 25292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:22.173051 25292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:22.177536 25292 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:46281
I20260808 22:59:22.177614 25363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:46281 every 8 connection(s)
I20260808 22:59:22.177973 25292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:22.178992 25364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:22.181557 25364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: Bootstrap starting.
I20260808 22:59:22.182224 25364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:22.182529 25364 log.cc:824] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:22.183398 25364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: No bootstrap required, opened a new log
I20260808 22:59:22.185222 25364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } }
I20260808 22:59:22.185380 25364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:22.185403 25364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57576cc8ecfd4219bfc70d8584f00d05, State: Initialized, Role: FOLLOWER
I20260808 22:59:22.185601 25364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } }
I20260808 22:59:22.185693 25364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:22.185745 25364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:22.185799 25364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:22.186755 25364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } }
I20260808 22:59:22.186877 25364 leader_election.cc:302] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57576cc8ecfd4219bfc70d8584f00d05; no voters:
I20260808 22:59:22.187085 25364 leader_election.cc:288] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:22.187136 25367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:22.187383 25367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [term 1 LEADER]: Becoming Leader. State: Replica: 57576cc8ecfd4219bfc70d8584f00d05, State: Running, Role: LEADER
I20260808 22:59:22.187393 25364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:22.187564 25367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } }
I20260808 22:59:22.187606 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25292
I20260808 22:59:22.187770 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:22.187978 1872 external_mini_cluster.cc:1428] Setting key 415c9855f27874ffbfe87e7d6cca830e
I20260808 22:59:22.188239 25369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57576cc8ecfd4219bfc70d8584f00d05. Latest consensus state: current_term: 1 leader_uuid: "57576cc8ecfd4219bfc70d8584f00d05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } } }
I20260808 22:59:22.188336 25369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:22.188724 25368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57576cc8ecfd4219bfc70d8584f00d05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57576cc8ecfd4219bfc70d8584f00d05" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 46281 } } }
I20260808 22:59:22.188963 25368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05 [sys.catalog]: This master's current role is: LEADER
W20260808 22:59:22.189471 25386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:22.189539 25386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:22.189641 25377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:22.189814 25377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:22.190986 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:22.188610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34729 (local address 127.1.212.62:46281)
0808 22:59:22.188835 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:22.188845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:22.188919 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:59:22.189167 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:22.189175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:22.189210 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:22.189360 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:22.190477 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:22.190481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:22.190482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:22.190487 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:22.190538 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:22.190554 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:22.190660 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:22.190736 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 22:59:22.190785 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260808 22:59:22.191744 25377 catalog_manager.cc:1347] Generated new cluster ID: 3cda010915e64e76bd2634f27c3d0d71
I20260808 22:59:22.191795 25377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:22.192008 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:22.194730 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:22.193542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50319 (local address 127.1.212.62:46281)
0808 22:59:22.193578 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:22.193580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:22.193735 (+ 155us) server_negotiation.cc:407] Connection header received
0808 22:59:22.193805 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:22.193806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:22.193811 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:22.193871 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:22.194507 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:22.194509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:22.194509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:22.194513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:22.194530 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:22.194535 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:22.194593 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:22.194646 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 22:59:22.194659 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 22:59:22.197399 25377 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:22.197613 25377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:22.212913 25377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57576cc8ecfd4219bfc70d8584f00d05: Generated new TSK 0
I20260808 22:59:22.213173 25377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:22.216162 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25292
2026-08-08T22:59:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:22 dist-test-slave-1x4g krb5kdc[25396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:22 dist-test-slave-1x4g krb5kdc[25396](info): set up 2 sockets
Aug 08 22:59:22 dist-test-slave-1x4g krb5kdc[25396](info): commencing operation
krb5kdc: starting...
W20260808 22:59:23.293494 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:23.433192 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43171
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:23.521598 25411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:23.521898 25411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:23.521943 25411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:23.523204 25411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:23.523288 25411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:23.523315 25411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:23.523347 25411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:23.523377 25411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:23.525023 25411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:43171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:23.525350 25411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:23.525609 25411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:23.531016 25416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:23.531098 25411 server_base.cc:1034] running on GCE node
W20260808 22:59:23.531037 25417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:23.531080 25419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:23.531575 25411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:23.531910 25411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:23.533058 25411 hybrid_clock.cc:648] HybridClock initialized: now 1786229963533042 us; error 32 us; skew 500 ppm
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229963, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:23.535756 25411 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:23.536299 25411 webserver.cc:460] Webserver started at http://127.1.212.62:44507/ using document root <none> and password file <none>
I20260808 22:59:23.536567 25411 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:23.536653 25411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:23.536862 25411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:23.538028 25411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "6fb69b00b60943809e61f3a21e51b3a4"
format_stamp: "Formatted at 2026-08-08 22:59:23 on dist-test-slave-1x4g"
server_key: "dc1bb21814132badd607eb403f36c386"
server_key_iv: "e1d2187ed1a25a6c1d79ffe3c1635580"
server_key_version: "encryptionkey@0"
I20260808 22:59:23.538422 25411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "6fb69b00b60943809e61f3a21e51b3a4"
format_stamp: "Formatted at 2026-08-08 22:59:23 on dist-test-slave-1x4g"
server_key: "dc1bb21814132badd607eb403f36c386"
server_key_iv: "e1d2187ed1a25a6c1d79ffe3c1635580"
server_key_version: "encryptionkey@0"
I20260808 22:59:23.540002 25411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:23.540863 25427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:23.541047 25411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:23.541159 25411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "6fb69b00b60943809e61f3a21e51b3a4"
format_stamp: "Formatted at 2026-08-08 22:59:23 on dist-test-slave-1x4g"
server_key: "dc1bb21814132badd607eb403f36c386"
server_key_iv: "e1d2187ed1a25a6c1d79ffe3c1635580"
server_key_version: "encryptionkey@0"
I20260808 22:59:23.541250 25411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:23.558429 25411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:23.560562 25411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:23.560750 25411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:23.565279 25411 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:43171
I20260808 22:59:23.565335 25483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:43171 every 8 connection(s)
I20260808 22:59:23.565752 25411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:23.566684 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:23.569090 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4: Bootstrap starting.
I20260808 22:59:23.569478 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25411
I20260808 22:59:23.569622 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:23.569839 1872 external_mini_cluster.cc:1428] Setting key f63198323e390187fc2dc16a151ce9ac
I20260808 22:59:23.569895 25484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:23.570253 25484 log.cc:824] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:23.571097 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4: No bootstrap required, opened a new log
I20260808 22:59:23.573186 25484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } }
I20260808 22:59:23.573367 25484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:23.573417 25484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb69b00b60943809e61f3a21e51b3a4, State: Initialized, Role: FOLLOWER
I20260808 22:59:23.573586 25484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } }
I20260808 22:59:23.573676 25484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:23.573724 25484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:23.573774 25484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:23.574388 25484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } }
I20260808 22:59:23.574523 25484 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb69b00b60943809e61f3a21e51b3a4; no voters:
I20260808 22:59:23.574750 25484 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:23.574776 25490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:23.574996 25490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [term 1 LEADER]: Becoming Leader. State: Replica: 6fb69b00b60943809e61f3a21e51b3a4, State: Running, Role: LEADER
I20260808 22:59:23.575111 25484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:23.575160 25490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } }
I20260808 22:59:23.575580 25491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fb69b00b60943809e61f3a21e51b3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } } }
I20260808 22:59:23.575681 25491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:23.575918 25497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:23.576509 25497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:23.578172 25492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb69b00b60943809e61f3a21e51b3a4. Latest consensus state: current_term: 1 leader_uuid: "6fb69b00b60943809e61f3a21e51b3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb69b00b60943809e61f3a21e51b3a4" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 43171 } } }
I20260808 22:59:23.578269 25492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:23.578915 25497 catalog_manager.cc:1347] Generated new cluster ID: 38227a34e2bc4ae386e96e1f2440cc5a
I20260808 22:59:23.578965 25497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:23.581548 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:23.570442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44237 (local address 127.1.212.62:43171)
0808 22:59:23.570696 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:23.570713 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:23.570740 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:59:23.571274 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:23.571282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:23.571337 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:23.571496 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:23.572168 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:23.573183 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:23.574165 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:23.574687 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:23.574806 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:23.577892 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:23.577896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:23.577896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:23.577908 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:23.579827 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:23.580235 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:23.580239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:23.580239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:23.580334 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:23.580566 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:23.580566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:23.580567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:23.580881 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:23.581047 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:23.581165 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 22:59:23.581312 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":126,"threads_started":1}
I20260808 22:59:23.582587 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:23.591392 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:23.584178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42747 (local address 127.1.212.62:43171)
0808 22:59:23.584221 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:23.584223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:23.584974 (+ 751us) server_negotiation.cc:407] Connection header received
0808 22:59:23.585052 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:23.585053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:23.585064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:23.585137 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:23.585703 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:23.586205 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:23.586947 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:23.587186 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:23.587266 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:23.588594 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:23.588595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:23.588595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:23.588603 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:23.590147 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:23.590560 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:23.590562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:23.590562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:23.590626 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:23.590834 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:23.590834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:23.590835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:23.590961 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:23.591081 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:23.591191 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 22:59:23.591279 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:23.611325 25497 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:23.612149 25497 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:23.618481 25497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fb69b00b60943809e61f3a21e51b3a4: Generated new TSK 0
I20260808 22:59:23.618750 25497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:23.622924 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25411
2026-08-08T22:59:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25518](info): set up 2 sockets
Aug 08 22:59:23 dist-test-slave-1x4g krb5kdc[25518](info): commencing operation
krb5kdc: starting...
W20260808 22:59:24.710331 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:24 dist-test-slave-1x4g krb5kdc[25518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:24.851634 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:42471
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:42471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:24.938532 25531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:24.938782 25531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:24.938809 25531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:24.940055 25531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:24.940109 25531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:24.940124 25531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:24.940137 25531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:24.940192 25531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:24.941802 25531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:42471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:24.942107 25531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:24.942380 25531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:24.947866 25537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:24.947891 25536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:24.947880 25539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:24.947919 25531 server_base.cc:1034] running on GCE node
I20260808 22:59:24.948464 25531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:24.948829 25531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:24.949999 25531 hybrid_clock.cc:648] HybridClock initialized: now 1786229964949987 us; error 33 us; skew 500 ppm
Aug 08 22:59:24 dist-test-slave-1x4g krb5kdc[25518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229964, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:24.956406 25531 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:24.957093 25531 webserver.cc:460] Webserver started at http://127.1.212.62:45219/ using document root <none> and password file <none>
I20260808 22:59:24.957304 25531 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:24.957391 25531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:24.957517 25531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:24.958601 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "f20b81d3110d4958889becc31c3f195b"
format_stamp: "Formatted at 2026-08-08 22:59:24 on dist-test-slave-1x4g"
server_key: "e79acf90a7e8f11a086d55c5c406cc13"
server_key_iv: "e2dbb946440fe8f0a965912e5a39da80"
server_key_version: "encryptionkey@0"
I20260808 22:59:24.958916 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "f20b81d3110d4958889becc31c3f195b"
format_stamp: "Formatted at 2026-08-08 22:59:24 on dist-test-slave-1x4g"
server_key: "e79acf90a7e8f11a086d55c5c406cc13"
server_key_iv: "e2dbb946440fe8f0a965912e5a39da80"
server_key_version: "encryptionkey@0"
I20260808 22:59:24.960284 25531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:24.961086 25547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:24.961256 25531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:24.961354 25531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "f20b81d3110d4958889becc31c3f195b"
format_stamp: "Formatted at 2026-08-08 22:59:24 on dist-test-slave-1x4g"
server_key: "e79acf90a7e8f11a086d55c5c406cc13"
server_key_iv: "e2dbb946440fe8f0a965912e5a39da80"
server_key_version: "encryptionkey@0"
I20260808 22:59:24.961414 25531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:24.983361 25531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:24.986017 25531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:24.986207 25531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:24.991350 25531 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:42471
I20260808 22:59:24.991425 25605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:42471 every 8 connection(s)
I20260808 22:59:24.991765 25531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:24.992717 25606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:24.995254 25606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: Bootstrap starting.
I20260808 22:59:24.995858 25606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:24.996120 25606 log.cc:824] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:24.996943 25606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: No bootstrap required, opened a new log
I20260808 22:59:24.998018 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25531
I20260808 22:59:24.998126 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:24.998332 1872 external_mini_cluster.cc:1428] Setting key cdb0e5ba8dc2db3022477fefee2ce639
I20260808 22:59:24.999048 25606 raft_consensus.cc:348] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } }
I20260808 22:59:24.999274 25606 raft_consensus.cc:374] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:24.999313 25606 raft_consensus.cc:729] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20b81d3110d4958889becc31c3f195b, State: Initialized, Role: FOLLOWER
I20260808 22:59:24.999445 25606 consensus_queue.cc:260] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } }
I20260808 22:59:24.999531 25606 raft_consensus.cc:388] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:24.999570 25606 raft_consensus.cc:482] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:24.999675 25606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:25.000321 25606 raft_consensus.cc:504] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } }
I20260808 22:59:25.000456 25606 leader_election.cc:302] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f20b81d3110d4958889becc31c3f195b; no voters:
I20260808 22:59:25.000689 25606 leader_election.cc:288] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:25.000722 25611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:25.000912 25611 raft_consensus.cc:686] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [term 1 LEADER]: Becoming Leader. State: Replica: f20b81d3110d4958889becc31c3f195b, State: Running, Role: LEADER
I20260808 22:59:25.001067 25606 sys_catalog.cc:564] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:25.001067 25611 consensus_queue.cc:237] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } }
I20260808 22:59:25.001500 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f20b81d3110d4958889becc31c3f195b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } } }
I20260808 22:59:25.001502 25613 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f20b81d3110d4958889becc31c3f195b. Latest consensus state: current_term: 1 leader_uuid: "f20b81d3110d4958889becc31c3f195b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20b81d3110d4958889becc31c3f195b" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 42471 } } }
I20260808 22:59:25.001678 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:25.001739 25613 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b [sys.catalog]: This master's current role is: LEADER
W20260808 22:59:25.003244 25628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:25.003358 25628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:25.003785 25619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:25.004071 25619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:25.004705 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:24.998944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49841 (local address 127.1.212.62:42471)
0808 22:59:24.999186 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:24.999196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:24.999232 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:59:24.999355 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:24.999362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:24.999436 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:24.999604 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:24.999850 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:25.001326 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:25.002422 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:25.003028 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:25.003160 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:25.003913 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:25.003917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:25.003918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:25.004156 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:25.004200 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:25.004215 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:25.004291 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:25.004386 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:59:25.004460 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260808 22:59:25.005843 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:25.009272 25619 catalog_manager.cc:1347] Generated new cluster ID: 4fc924d1463d4e3f8d549debf6d8e692
I20260808 22:59:25.009339 25619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:25.010860 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:25.007437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37755 (local address 127.1.212.62:42471)
0808 22:59:25.007482 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:25.007483 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:25.007604 (+ 121us) server_negotiation.cc:407] Connection header received
0808 22:59:25.007636 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:25.007637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:25.007665 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:25.007737 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:25.007843 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:25.008603 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:25.009498 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:25.009826 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:25.009904 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:25.010599 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:25.010601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:25.010602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:25.010608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:25.010625 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:25.010629 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:25.010691 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:25.010752 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 22:59:25.010781 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:59:25.017865 25619 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:25.018636 25619 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:25.023362 25619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f20b81d3110d4958889becc31c3f195b: Generated new TSK 0
I20260808 22:59:25.023610 25619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:25.030331 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25531
2026-08-08T22:59:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:25 dist-test-slave-1x4g krb5kdc[25639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:25 dist-test-slave-1x4g krb5kdc[25639](info): set up 2 sockets
Aug 08 22:59:25 dist-test-slave-1x4g krb5kdc[25639](info): commencing operation
krb5kdc: starting...
W20260808 22:59:26.113833 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:26 dist-test-slave-1x4g krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:26.253305 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34783
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:34783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:26.342494 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:26.342804 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:26.342849 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:26.344188 25652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:26.344285 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:26.344311 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:26.344334 25652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:26.344365 25652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:26.346235 25652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:34783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:34783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:26.346577 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:26.346849 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:26.352273 25657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:26.352296 25658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:26.352273 25661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:26.352610 25652 server_base.cc:1034] running on GCE node
I20260808 22:59:26.352912 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:26.353253 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:26.354424 25652 hybrid_clock.cc:648] HybridClock initialized: now 1786229966354410 us; error 31 us; skew 500 ppm
Aug 08 22:59:26 dist-test-slave-1x4g krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229966, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:26.357409 25652 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:26.357945 25652 webserver.cc:460] Webserver started at http://127.1.212.62:45425/ using document root <none> and password file <none>
I20260808 22:59:26.358235 25652 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:26.358314 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:26.358477 25652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:26.359724 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "95b8bca6a14f46dc93abe50115be1400"
format_stamp: "Formatted at 2026-08-08 22:59:26 on dist-test-slave-1x4g"
server_key: "5a8d23b9031bc51d2afcf8ded9767296"
server_key_iv: "302e4897ac6de1c42018da0c3fe9fc71"
server_key_version: "encryptionkey@0"
I20260808 22:59:26.360106 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "95b8bca6a14f46dc93abe50115be1400"
format_stamp: "Formatted at 2026-08-08 22:59:26 on dist-test-slave-1x4g"
server_key: "5a8d23b9031bc51d2afcf8ded9767296"
server_key_iv: "302e4897ac6de1c42018da0c3fe9fc71"
server_key_version: "encryptionkey@0"
I20260808 22:59:26.361666 25652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:26.362480 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:26.362684 25652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:26.362799 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "95b8bca6a14f46dc93abe50115be1400"
format_stamp: "Formatted at 2026-08-08 22:59:26 on dist-test-slave-1x4g"
server_key: "5a8d23b9031bc51d2afcf8ded9767296"
server_key_iv: "302e4897ac6de1c42018da0c3fe9fc71"
server_key_version: "encryptionkey@0"
I20260808 22:59:26.362881 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:26.366524 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:26.368701 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:26.368906 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:26.373433 25652 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:34783
I20260808 22:59:26.373492 25723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:34783 every 8 connection(s)
I20260808 22:59:26.373880 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:26.374993 25724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:26.377641 25724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400: Bootstrap starting.
I20260808 22:59:26.378261 25724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:26.378535 25724 log.cc:824] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:26.379330 25724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400: No bootstrap required, opened a new log
I20260808 22:59:26.379890 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25652
I20260808 22:59:26.380035 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:26.380236 1872 external_mini_cluster.cc:1428] Setting key 70a709932931ef3700d6d2f4f35c58bc
I20260808 22:59:26.382237 25724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } }
I20260808 22:59:26.382539 25724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:26.382589 25724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b8bca6a14f46dc93abe50115be1400, State: Initialized, Role: FOLLOWER
I20260808 22:59:26.383266 25724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } }
I20260808 22:59:26.383400 25724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:26.383497 25729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:26.383608 25729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } }
I20260808 22:59:26.383759 25729 leader_election.cc:302] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b8bca6a14f46dc93abe50115be1400; no voters:
I20260808 22:59:26.383814 25724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:26.383834 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:26.380890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33001 (local address 127.1.212.62:34783)
0808 22:59:26.381094 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:26.381104 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:26.381135 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:59:26.382124 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:26.382134 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:26.382170 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:26.382317 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:26.383172 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:26.383175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:26.383176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:26.383180 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:26.383221 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:26.383232 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:26.383318 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:26.383425 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 22:59:26.383474 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260808 22:59:26.383850 25724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:26.383971 25729 leader_election.cc:288] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260808 22:59:26.384661 25724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } }
I20260808 22:59:26.384737 25724 leader_election.cc:302] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b8bca6a14f46dc93abe50115be1400; no voters:
I20260808 22:59:26.384766 25724 leader_election.cc:288] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:26.384835 25730 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 22:59:26.384989 25729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:26.385084 25729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [term 1 LEADER]: Becoming Leader. State: Replica: 95b8bca6a14f46dc93abe50115be1400, State: Running, Role: LEADER
I20260808 22:59:26.385133 25724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:26.385244 25729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } }
I20260808 22:59:26.385643 25730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95b8bca6a14f46dc93abe50115be1400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } } }
I20260808 22:59:26.385742 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95b8bca6a14f46dc93abe50115be1400. Latest consensus state: current_term: 1 leader_uuid: "95b8bca6a14f46dc93abe50115be1400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8bca6a14f46dc93abe50115be1400" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 34783 } } }
I20260808 22:59:26.385752 25730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:26.385879 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:26.386096 25738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:26.386339 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:26.386737 25738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:26.388659 25738 catalog_manager.cc:1347] Generated new cluster ID: d80dd0dbd5f54fe59a63b783a84dd628
I20260808 22:59:26.388716 25738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:26.389370 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:26.387997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37439 (local address 127.1.212.62:34783)
0808 22:59:26.388041 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:26.388044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:26.388194 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:59:26.388253 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:26.388254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:26.388258 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:26.388337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:26.389141 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:26.389143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:26.389143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:26.389147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:26.389165 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:26.389169 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:26.389230 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:26.389295 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 22:59:26.389308 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:26.401197 25738 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:26.401455 25738 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:26.411397 25738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95b8bca6a14f46dc93abe50115be1400: Generated new TSK 0
I20260808 22:59:26.411654 25738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:26.419452 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25652
2026-08-08T22:59:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1388 ms)
[----------] 18 tests from AuthTokenIssuingTest (25164 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:26 dist-test-slave-1x4g krb5kdc[25757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:26 dist-test-slave-1x4g krb5kdc[25757](info): set up 2 sockets
Aug 08 22:59:26 dist-test-slave-1x4g krb5kdc[25757](info): commencing operation
krb5kdc: starting...
W20260808 22:59:27.498394 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:27.637163 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40065
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:27.722862 25771 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:27.723168 25771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:27.723202 25771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:27.723266 25771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:27.724545 25771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:27.724637 25771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:27.724660 25771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:27.724679 25771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:27.724712 25771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:27.726449 25771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:40065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:27.726783 25771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:27.727062 25771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:27.732301 25777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:27.732331 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:27.732347 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:27.732599 25771 server_base.cc:1034] running on GCE node
I20260808 22:59:27.732919 25771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:27.733289 25771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:27.734504 25771 hybrid_clock.cc:648] HybridClock initialized: now 1786229967734487 us; error 47 us; skew 500 ppm
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:27.737299 25771 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:27.737802 25771 webserver.cc:460] Webserver started at http://127.1.212.62:33223/ using document root <none> and password file <none>
I20260808 22:59:27.738073 25771 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:27.738147 25771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:27.738312 25771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:27.739444 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "9a8c502dd7ca4d79b4f753475089a923"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "f0840cae68560314bd95846c8e0f7758"
server_key_iv: "7b2f428e29327aaf47002fecc5f91ffe"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.739827 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "9a8c502dd7ca4d79b4f753475089a923"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "f0840cae68560314bd95846c8e0f7758"
server_key_iv: "7b2f428e29327aaf47002fecc5f91ffe"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.741343 25771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:27.742136 25786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:27.742445 25771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:27.742570 25771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "9a8c502dd7ca4d79b4f753475089a923"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "f0840cae68560314bd95846c8e0f7758"
server_key_iv: "7b2f428e29327aaf47002fecc5f91ffe"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.742660 25771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:27.757615 25771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:27.759675 25771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:27.759860 25771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:27.764565 25771 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:40065
I20260808 22:59:27.764621 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:40065 every 8 connection(s)
I20260808 22:59:27.765055 25771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:27.766248 25841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:27.768038 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:27.773823 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25771
I20260808 22:59:27.773928 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:27.774134 1872 external_mini_cluster.cc:1428] Setting key daae2684427c293e97bfae46a4255d72
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:27.779701 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.768877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40065 (local address 127.0.0.1:43656)
0808 22:59:27.769091 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.769166 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:27.769706 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:27.770027 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:27.770030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:27.770357 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:27.770524 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.770529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.771596 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.771598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.772297 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.772299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.772879 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.772880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.772949 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.773584 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:27.773595 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:27.775538 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:27.777935 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.777943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.777945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.778186 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.778484 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.778484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.778484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.778574 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.779116 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:27.779123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:27.779188 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 22:59:27.779271 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:59:27.779515 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260808 22:59:27.779798 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.769066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.1.212.62:40065)
0808 22:59:27.769195 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:27.769200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:27.769262 (+ 62us) server_negotiation.cc:407] Connection header received
0808 22:59:27.769901 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:27.769903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:27.769922 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:27.770021 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:27.770645 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.771491 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.772402 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.772727 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.772907 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.775642 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:27.775651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:27.775651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:27.775659 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:27.777821 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.778282 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.778286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.778287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.778407 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.778643 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.778644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.778644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.779029 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:27.779140 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:27.779345 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 22:59:27.779623 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260808 22:59:27.780715 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:27.781157 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.781027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37480)
0808 22:59:27.781081 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.781135 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:59:27.781342 25789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
I20260808 22:59:27.781805 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.781711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37492)
0808 22:59:27.781763 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.781788 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:59:27.781944 25841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:40465: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.212.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
I20260808 22:59:27.805428 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:40465
--webserver_interface=127.1.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:59:27.831195 25841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } attempt: 1
I20260808 22:59:27.831744 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.831597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37508)
0808 22:59:27.831671 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.831712 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:59:27.832109 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.832008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37514)
0808 22:59:27.832056 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.832084 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:59:27.832199 25841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:40465: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
W20260808 22:59:27.891021 25848 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:27.891315 25848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:27.891350 25848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:27.891395 25848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:27.892660 25848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:27.892712 25848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:27.892765 25848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:27.892784 25848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:27.892830 25848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:27.894510 25848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:40465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:27.894845 25848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:27.895138 25848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:27.900363 25841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } attempt: 2
W20260808 22:59:27.900406 25853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:27.900406 25854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:27.900797 25848 server_base.cc:1034] running on GCE node
W20260808 22:59:27.900838 25856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:27.900878 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.900699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37518)
0808 22:59:27.900793 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.900833 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:59:27.901176 25848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:27.901271 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.901144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37534)
0808 22:59:27.901203 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.901240 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 22:59:27.901381 25841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:40465: Network error: Client connection negotiation failed: client connection to 127.1.212.61:40465: connect: Connection refused (error 111)
I20260808 22:59:27.901556 25848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:27.902711 25848 hybrid_clock.cc:648] HybridClock initialized: now 1786229967902693 us; error 31 us; skew 500 ppm
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:27.905400 25848 init.cc:377] Logged in from keytab as kudu/127.1.212.61@KRBTEST.COM (short username kudu)
I20260808 22:59:27.905869 25848 webserver.cc:460] Webserver started at http://127.1.212.61:38593/ using document root <none> and password file <none>
I20260808 22:59:27.906109 25848 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:27.906180 25848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:27.906340 25848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:27.907478 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data/instance:
uuid: "989d9a7555e74a63884fd1591bf4f22f"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "32308a70afabc765f8cd158b282c7f69"
server_key_iv: "16f3a3bf39b02b7ba70c182620fbad0b"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.907864 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal/instance:
uuid: "989d9a7555e74a63884fd1591bf4f22f"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "32308a70afabc765f8cd158b282c7f69"
server_key_iv: "16f3a3bf39b02b7ba70c182620fbad0b"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.909425 25848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:27.910187 25863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:27.910364 25848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 22:59:27.910485 25848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "989d9a7555e74a63884fd1591bf4f22f"
format_stamp: "Formatted at 2026-08-08 22:59:27 on dist-test-slave-1x4g"
server_key: "32308a70afabc765f8cd158b282c7f69"
server_key_iv: "16f3a3bf39b02b7ba70c182620fbad0b"
server_key_version: "encryptionkey@0"
I20260808 22:59:27.910570 25848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:27.923969 25848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:27.926198 25848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:27.926375 25848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:27.930692 25848 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.61:40465
I20260808 22:59:27.930744 25919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.61:40465 every 8 connection(s)
I20260808 22:59:27.931135 25848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
I20260808 22:59:27.931607 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25848
I20260808 22:59:27.931727 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-1/wal/instance
I20260808 22:59:27.931905 1872 external_mini_cluster.cc:1428] Setting key 181aa05a8581ed4fd2e73fa102065543
I20260808 22:59:27.932307 25920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:27.934036 25920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.212.60@KRBTEST.COM; defaulting to no policy
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.60@KRBTEST.COM" created.
I20260808 22:59:27.945541 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.934819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.1.212.62:40065)
0808 22:59:27.934911 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:27.934913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:27.935150 (+ 237us) server_negotiation.cc:407] Connection header received
0808 22:59:27.935952 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:27.935954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:27.935965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:27.936009 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:27.937048 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.937567 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.939285 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.939540 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.939602 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.942215 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:27.942217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:27.942217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:27.942225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:27.943610 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.944098 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.944099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.944099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.944294 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.944670 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.944671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.944671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.944875 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:27.944976 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:27.945274 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 22:59:27.945375 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:59:27.945681 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.934776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40065 (local address 127.0.0.1:43660)
0808 22:59:27.934995 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.935047 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:27.935820 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:27.936084 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:27.936089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:27.936551 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:27.936929 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.936937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.937686 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.937687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.939189 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.939193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.939649 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.939651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.939832 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.940387 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:27.940399 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:27.942112 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:27.943712 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.943727 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.943730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.943997 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.944421 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.944422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.944422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.944544 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.944979 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:27.944988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:27.945134 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 22:59:27.945250 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 22:59:27.945427 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
I20260808 22:59:27.946424 25920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:27.955299 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.946719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37548)
0808 22:59:27.946780 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.946791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:27.946828 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:27.947203 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:27.947204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:27.947392 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:27.947465 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.947467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.948595 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.948596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.949174 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.949176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.949949 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.949950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.950023 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.950464 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:27.950469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:27.951705 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:27.954170 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.954171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.954172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.954350 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.954614 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.954615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.954615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.954673 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.955010 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:27.955011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:27.955051 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:59:27.955111 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:59:27.955204 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:59:27.955399 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.946809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.1.212.61:40465)
0808 22:59:27.947049 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:27.947054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:27.947071 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:59:27.947092 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:27.947094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:27.947105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:27.947173 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:27.947545 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.948502 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.949265 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.949842 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.949931 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.951801 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:27.951811 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:27.951811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:27.951820 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:27.954070 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.954446 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.954450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.954450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.954525 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.954745 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.954746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.954746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.954946 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:27.955027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:27.955147 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 22:59:27.955255 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":196,"thread_start_us":156,"threads_started":1}
I20260808 22:59:27.956094 25920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:27.956485 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.956353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47696)
0808 22:59:27.956406 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.956463 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:59:27.956642 25868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
I20260808 22:59:27.957432 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.957311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47702)
0808 22:59:27.957369 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.957396 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:59:27.957556 25920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:41437: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
I20260808 22:59:27.963922 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:41437
--webserver_interface=127.1.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:59:27.992558 25841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } attempt: 3
Aug 08 22:59:27 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:28.000492 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.993017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37556)
0808 22:59:27.993082 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:27.993099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:27.993159 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:27.993323 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:27.993324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:27.993504 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:27.993588 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.993590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.994289 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.994291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.994854 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:27.994856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.995296 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.995297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:27.995398 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.995940 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:27.995946 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:27.997294 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:27.999050 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.999051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.999052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.999336 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:27.999606 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:27.999607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:27.999608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:27.999690 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.000059 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.000061 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.000159 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.000252 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.000373 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 22:59:28.001004 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:27.993109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.1.212.61:40465)
0808 22:59:27.993184 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:27.993186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:27.993197 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:59:27.993235 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:27.993235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:27.993247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:27.993315 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:27.993680 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.994197 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.994947 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:27.995203 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:27.995285 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:27.997395 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:27.997397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:27.997397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:27.997414 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:27.998906 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.999451 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.999452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.999453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.999523 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:27.999790 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:27.999791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:27.999791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:27.999945 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.000022 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.000793 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.000902 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:59:28.001360 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:28.001820 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.001718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47718)
0808 22:59:28.001768 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.001795 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 22:59:28.002218 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.002047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47728)
0808 22:59:28.002174 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.002194 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
W20260808 22:59:28.002288 25841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:41437: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
I20260808 22:59:28.006829 25920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } attempt: 1
I20260808 22:59:28.007359 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.007219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47730)
0808 22:59:28.007283 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.007328 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 22:59:28.007742 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.007651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47746)
0808 22:59:28.007693 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.007722 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 22:59:28.007869 25920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:41437: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
I20260808 22:59:28.033445 25841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } attempt: 1
I20260808 22:59:28.034050 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.033870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47754)
0808 22:59:28.033951 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.034007 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:59:28.034451 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.034336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47768)
0808 22:59:28.034382 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.034425 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 22:59:28.034561 25841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:41437: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
W20260808 22:59:28.053575 25929 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:28.053829 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:28.053848 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:28.053870 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:28.055087 25929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:28.055135 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:28.055148 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:28.055161 25929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:28.055174 25929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:28.056919 25929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:40065,127.1.212.61:40465,127.1.212.60:41437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:41437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:28.057173 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:28.057379 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:28.062657 25938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:28.062675 25936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:28.062783 25935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:28.062889 25929 server_base.cc:1034] running on GCE node
I20260808 22:59:28.063190 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:28.063555 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:28.064747 25929 hybrid_clock.cc:648] HybridClock initialized: now 1786229968064736 us; error 51 us; skew 500 ppm
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:28.067768 25929 init.cc:377] Logged in from keytab as kudu/127.1.212.60@KRBTEST.COM (short username kudu)
I20260808 22:59:28.068305 25929 webserver.cc:460] Webserver started at http://127.1.212.60:41065/ using document root <none> and password file <none>
I20260808 22:59:28.068564 25929 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:28.068634 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:28.068849 25929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:28.070111 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data/instance:
uuid: "ede2eba75bc344d9adf5f7c8a8e39057"
format_stamp: "Formatted at 2026-08-08 22:59:28 on dist-test-slave-1x4g"
server_key: "827cb5704cb6038c95a612b534b992f3"
server_key_iv: "b725a4497c640891881fe9c93414bc21"
server_key_version: "encryptionkey@0"
I20260808 22:59:28.070520 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal/instance:
uuid: "ede2eba75bc344d9adf5f7c8a8e39057"
format_stamp: "Formatted at 2026-08-08 22:59:28 on dist-test-slave-1x4g"
server_key: "827cb5704cb6038c95a612b534b992f3"
server_key_iv: "b725a4497c640891881fe9c93414bc21"
server_key_version: "encryptionkey@0"
I20260808 22:59:28.072023 25929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:28.072903 25945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:28.073114 25929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:28.073230 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "ede2eba75bc344d9adf5f7c8a8e39057"
format_stamp: "Formatted at 2026-08-08 22:59:28 on dist-test-slave-1x4g"
server_key: "827cb5704cb6038c95a612b534b992f3"
server_key_iv: "b725a4497c640891881fe9c93414bc21"
server_key_version: "encryptionkey@0"
I20260808 22:59:28.073313 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:28.076063 25920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } attempt: 2
I20260808 22:59:28.076606 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.076412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47778)
0808 22:59:28.076490 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.076550 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:59:28.077064 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.076943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47794)
0808 22:59:28.076996 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.077027 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:59:28.077178 25920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:41437: Network error: Client connection negotiation failed: client connection to 127.1.212.60:41437: connect: Connection refused (error 111)
I20260808 22:59:28.086602 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:28.089239 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:28.089371 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:28.093762 25929 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.60:41437
I20260808 22:59:28.093802 26000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.60:41437 every 8 connection(s)
I20260808 22:59:28.094218 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
I20260808 22:59:28.095358 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:28.097404 26001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:28.100328 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 25929
I20260808 22:59:28.100430 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229688278322-1872-0/minicluster-data/master-2/wal/instance
I20260808 22:59:28.100631 1872 external_mini_cluster.cc:1428] Setting key a8569f5a669c29a6bf8c389f1e93b8d9
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:28.108987 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.098256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43676 (local address 127.1.212.62:40065)
0808 22:59:28.098376 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.098379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.098597 (+ 218us) server_negotiation.cc:407] Connection header received
0808 22:59:28.099145 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.099147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.099158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.099223 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.099910 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.100471 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.101870 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.102082 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.102179 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.104665 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.104667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.104667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.104676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.107254 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.107727 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.107728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.107729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.107793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.108064 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.108065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.108065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.108199 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.108278 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.108666 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.108817 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 22:59:28.109063 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.098213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:40065 (local address 127.0.0.1:43676)
0808 22:59:28.098457 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.098499 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:28.098996 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:28.099315 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:28.099319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:28.099596 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:28.099808 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.099812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.100606 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.100608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.101761 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.101765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.102206 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.102207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.102348 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.102927 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:28.102937 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:28.104562 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:28.107350 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.107358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.107361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.107628 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.107868 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.107868 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.107869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.107994 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.108268 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.108275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.108474 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.108599 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.108815 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":146,"threads_started":1}
I20260808 22:59:28.109689 25841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } attempt: 2
I20260808 22:59:28.109913 26001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:28.110625 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.101135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.1.212.60:41437)
0808 22:59:28.101347 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.101352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.101396 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:59:28.101484 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.101488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.101511 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.101602 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.102243 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.102964 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.103875 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.104266 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.104326 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.106784 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.106796 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.106797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.106807 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.108554 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.109203 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.109206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.109207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.109386 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.109868 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.109870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.109870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.110063 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.110220 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.110389 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.110486 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:28.119153 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.110187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:40465 (local address 127.0.0.1:37560)
0808 22:59:28.110292 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.110301 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:28.110336 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:28.110493 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:28.110494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:28.110631 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:28.110710 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.110712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.111434 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.111435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.112255 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.112257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.112795 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.112797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.112880 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.113301 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:28.113307 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:28.115410 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:28.117904 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.117904 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.117905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.118110 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.118349 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.118350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.118416 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.118799 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.118800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.118848 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.118921 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.119034 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 22:59:28.119194 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.110305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.1.212.61:40465)
0808 22:59:28.110355 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.110357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.110383 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:59:28.110409 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.110410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.110421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.110476 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.110822 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.111321 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.112341 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.112674 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.112764 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.115498 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.115499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.115500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.115507 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.117798 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118198 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.118200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.118200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.118271 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118515 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.118516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.118516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.118685 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.118747 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.118989 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.119079 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:59:28.119365 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.109948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47822)
0808 22:59:28.109996 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.110007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:28.110044 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:28.110860 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:28.110861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:28.111017 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:28.111064 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.111067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.111916 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.111917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.112550 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.112552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.113083 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.113084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.113142 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.113512 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:28.113517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:28.115328 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:28.118084 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.118086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.118397 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.118604 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.118605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.118678 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.118974 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.118975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.119028 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.119107 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.119231 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 22:59:28.119752 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.110136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.1.212.60:41437)
0808 22:59:28.110725 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.110727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.110735 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:28.110757 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.110758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.110767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.111140 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.111172 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.111840 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.112637 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.112974 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.113033 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.115430 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.115431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.115431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.115439 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.118002 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118475 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.118476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.118477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.118530 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.118753 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.118754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.118754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.118879 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.118941 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.119570 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.119657 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":131,"threads_started":1}
I20260808 22:59:28.120070 26001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:28.121178 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.111444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.1.212.62:40065)
0808 22:59:28.111484 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.111486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.111494 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:28.111583 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.111584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.111594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.111643 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.112239 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.112858 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.113884 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.114181 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.114216 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.117055 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.117057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.117057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.117065 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.119468 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.119917 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.119919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.119919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.120171 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.120416 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.120417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.120417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.120580 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.120723 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.120979 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.121066 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:28.121708 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: Bootstrap starting.
I20260808 22:59:28.122475 25841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:28.122809 25841 log.cc:824] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:28.123968 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: No bootstrap required, opened a new log
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:28.126297 25841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.126461 25841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8c502dd7ca4d79b4f753475089a923, State: Initialized, Role: FOLLOWER
I20260808 22:59:28.126559 25841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:28.126924 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.126997 25841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:28.127493 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.128170 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.120402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.1.212.60:41437)
0808 22:59:28.120437 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.120439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.120475 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:59:28.120537 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.120538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.120548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.120609 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.120905 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.121593 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.122377 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.122685 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.122762 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.124901 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.124903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.124903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.124910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.126802 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.127231 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.127232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.127233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.127284 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.127459 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.127460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.127460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.127664 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.127722 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.127937 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.128020 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 22:59:28.128525 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.120385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47836)
0808 22:59:28.120432 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.120443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:28.120488 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:28.120611 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:28.120612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:28.120764 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:28.120840 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.120842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.121689 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.121690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.122282 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.122285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.122816 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.122817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.122865 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.123345 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:28.123350 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:28.124769 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:28.126898 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.126898 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.126899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.127130 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.127348 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.127349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.127349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.127398 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.127774 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.127775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.127822 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.128311 (+ 489us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.128425 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 22:59:28.129120 26019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:28.129169 26019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:28.130192 26001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: Bootstrap starting.
I20260808 22:59:28.131258 26001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:28.131584 26001 log.cc:824] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:28.132304 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.121825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.1.212.61:40465)
0808 22:59:28.121893 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.121895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.122433 (+ 538us) server_negotiation.cc:407] Connection header received
0808 22:59:28.122515 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.122516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.122527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.122583 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.123099 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.123753 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.124544 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.124746 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.124896 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.127484 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.127485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.127486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.127496 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.130199 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.130832 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.130833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.130833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.130915 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.131167 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.131168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.131168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.131424 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.132079 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.132127 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.132211 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:59:28.133523 26001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: No bootstrap required, opened a new log
I20260808 22:59:28.130157 25771 master_runner.cc:186] Error getting master registration for 127.1.212.61:40465: OK, instance_id { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" instance_seqno: 1786229967929426 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:59:28.133713 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:28.133749 25771 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:59:28.134820 25814 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9a8c502dd7ca4d79b4f753475089a923"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43680
I20260808 22:59:28.134967 25814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:59:28.135021 25814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:28.135349 26001 raft_consensus.cc:348] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.135505 26001 raft_consensus.cc:729] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede2eba75bc344d9adf5f7c8a8e39057, State: Initialized, Role: FOLLOWER
I20260808 22:59:28.135600 26001 consensus_queue.cc:260] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.135924 26022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.135941 25814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.136000 26001 sys_catalog.cc:564] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:28.136093 26022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.136206 25814 leader_election.cc:288] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [CANDIDATE]: Term 1 election: Requested vote from peers 989d9a7555e74a63884fd1591bf4f22f (127.1.212.61:40465), ede2eba75bc344d9adf5f7c8a8e39057 (127.1.212.60:41437)
I20260808 22:59:28.136627 25973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a8c502dd7ca4d79b4f753475089a923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ede2eba75bc344d9adf5f7c8a8e39057"
I20260808 22:59:28.136929 25973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:28.137861 25973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8c502dd7ca4d79b4f753475089a923 in term 1.
I20260808 22:59:28.138540 25791 leader_election.cc:302] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8c502dd7ca4d79b4f753475089a923, ede2eba75bc344d9adf5f7c8a8e39057; no voters:
I20260808 22:59:28.138706 26008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:28.138847 26008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [term 1 LEADER]: Becoming Leader. State: Replica: 9a8c502dd7ca4d79b4f753475089a923, State: Running, Role: LEADER
I20260808 22:59:28.138939 26008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.139693 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8c502dd7ca4d79b4f753475089a923. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.139957 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:28.140290 26041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:28.140715 26041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 22:59:28.140764 26037 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:28.140877 26037 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:28.141271 25929 master_runner.cc:186] Error getting master registration for 127.1.212.61:40465: OK, instance_id { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" instance_seqno: 1786229967929426 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:59:28.144304 25929 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:59:28.145931 25973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [term 1 FOLLOWER]: Refusing update from remote peer 9a8c502dd7ca4d79b4f753475089a923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:28.146294 26008 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:28.148734 26022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8c502dd7ca4d79b4f753475089a923. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.148948 26022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.157557 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.145145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.1.212.62:40065)
0808 22:59:28.145203 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.145206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.145875 (+ 669us) server_negotiation.cc:407] Connection header received
0808 22:59:28.145971 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.145972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.145985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.146046 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.147555 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.148343 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.150723 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.151041 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.151097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.153809 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.153812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.153812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.153822 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.155645 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.156286 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.156288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.156288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.156354 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.156699 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.156701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.156701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.156884 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.156968 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.157352 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.157453 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:59:28.158489 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.158625 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:28.158834 26043 mvcc.cc:205] Tried to move back new op lower bound from 7316397949522534400 to 7316397949497929728. Current Snapshot: MvccSnapshot[applied={T|T < 7316397949522534400}]
I20260808 22:59:28.159237 26022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.159353 26022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.162359 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.145377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37582 (local address 127.1.212.61:40465)
0808 22:59:28.145426 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.145428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.152954 (+ 7526us) server_negotiation.cc:407] Connection header received
0808 22:59:28.153053 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.153054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.153070 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.153171 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.153828 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.155228 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.156366 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.156700 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.156843 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.158924 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.158926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.158926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.158935 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.160988 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.161423 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.161425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.161425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.161492 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.161710 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.161711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.161711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.161852 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.161931 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.162150 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.162253 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:28.166254 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.145654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.1.212.60:41437)
0808 22:59:28.145700 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.145702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.153161 (+ 7459us) server_negotiation.cc:407] Connection header received
0808 22:59:28.153917 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.153918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.153931 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.153962 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.154669 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.155383 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.157298 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.157622 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.157695 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.159876 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.159878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.159878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.159886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.161641 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.165277 (+ 3636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.165278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.165279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.165380 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.165610 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.165611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.165611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.165768 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.165889 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.166057 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.166154 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:28.168275 25920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } attempt: 3
I20260808 22:59:28.173045 26041 catalog_manager.cc:1347] Generated new cluster ID: 794ab1cadebb43009db27f2ef075d205
I20260808 22:59:28.173115 26041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:28 dist-test-slave-1x4g krb5kdc[25757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:28.178345 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.168594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:41437 (local address 127.0.0.1:47848)
0808 22:59:28.168696 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:28.168709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:28.168748 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:28.168931 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:28.168932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:28.169110 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:28.169165 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.169166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.170148 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.170150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.170978 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:28.170981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.171537 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.171538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:28.171598 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.172145 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:28.172153 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:28.173890 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:28.175898 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.175899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.175900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.176202 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.176450 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:28.176451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:28.176451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:28.176517 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:28.176969 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:28.176971 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:28.177026 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 22:59:28.177094 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:59:28.177227 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 22:59:28.178799 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:28.168704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.1.212.60:41437)
0808 22:59:28.168768 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.168794 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.168823 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:59:28.168846 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.168847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.168857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.168909 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.169238 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.170027 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.171091 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:28.171409 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:28.171472 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:28.173991 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:28.173993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:28.173993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:28.174001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:28.175770 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.176282 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.176283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.176283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.176382 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:28.176572 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:28.176573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:28.176574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:28.176858 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:28.176936 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:28.178597 (+ 1661us) server_negotiation.cc:299] Negotiation successful
0808 22:59:28.178702 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 22:59:28.180540 25920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: Bootstrap starting.
I20260808 22:59:28.181458 25920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:28.181793 25920 log.cc:824] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:28.182895 25920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: No bootstrap required, opened a new log
I20260808 22:59:28.186215 25920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.186426 25920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989d9a7555e74a63884fd1591bf4f22f, State: Initialized, Role: FOLLOWER
I20260808 22:59:28.186527 25920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } }
I20260808 22:59:28.186944 25920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:28.187047 25891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:28.188086 25891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [term 1 FOLLOWER]: Refusing update from remote peer 9a8c502dd7ca4d79b4f753475089a923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:28.188618 26008 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:28.188311 25892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a8c502dd7ca4d79b4f753475089a923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "989d9a7555e74a63884fd1591bf4f22f"
I20260808 22:59:28.189952 25892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8c502dd7ca4d79b4f753475089a923 in term 1.
I20260808 22:59:28.192015 26041 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:28.193136 26041 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:28.194291 26065 mvcc.cc:205] Tried to move back new op lower bound from 7316397949522534400 to 7316397949497929728. Current Snapshot: MvccSnapshot[applied={T|T < 7316397949522534400}]
I20260808 22:59:28.195178 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.195174 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.195288 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:28.195291 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.195343 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8c502dd7ca4d79b4f753475089a923. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.195384 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:28.195430 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a8c502dd7ca4d79b4f753475089a923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8c502dd7ca4d79b4f753475089a923" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 40065 } } peers { permanent_uuid: "989d9a7555e74a63884fd1591bf4f22f" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 40465 } } peers { permanent_uuid: "ede2eba75bc344d9adf5f7c8a8e39057" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 41437 } } }
I20260808 22:59:28.195480 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:59:28.196696 26063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:28.196767 26063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:28.202836 26041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a8c502dd7ca4d79b4f753475089a923: Generated new TSK 0
I20260808 22:59:28.203071 26041 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:59:28.232151 26005 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.230341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.1.212.60:41437)
0808 22:59:28.230393 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.230395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.231410 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 22:59:28.231528 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.231529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.231542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.231585 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.232066 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47862: BlockingRecv error: recv got EOF from 127.0.0.1:47862 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 22:59:28.233033 25844 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.230793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.1.212.62:40065)
0808 22:59:28.230843 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.230845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.232427 (+ 1582us) server_negotiation.cc:407] Connection header received
0808 22:59:28.232505 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.232506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.232520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.232579 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.232954 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43686: BlockingRecv error: recv got EOF from 127.0.0.1:43686 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 22:59:28.233232 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.231088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.1.212.61:40465)
0808 22:59:28.231131 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.231133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.232603 (+ 1470us) server_negotiation.cc:407] Connection header received
0808 22:59:28.232706 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.232707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.232720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.232750 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.233169 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37588: BlockingRecv error: recv got EOF from 127.0.0.1:37588 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 22:59:28.238322 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.237399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.1.212.61:40465)
0808 22:59:28.237442 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.237444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.237706 (+ 262us) server_negotiation.cc:407] Connection header received
0808 22:59:28.237829 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.237830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.237852 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.237883 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.238263 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37602: BlockingRecv error: recv got EOF from 127.0.0.1:37602 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 22:59:28.241660 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.240760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.1.212.61:40465)
0808 22:59:28.240825 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.240827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.241097 (+ 270us) server_negotiation.cc:407] Connection header received
0808 22:59:28.241197 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.241198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.241209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.241237 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.241604 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37606: BlockingRecv error: recv got EOF from 127.0.0.1:37606 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 22:59:28.245913 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.245032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.1.212.61:40465)
0808 22:59:28.245071 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.245074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.245351 (+ 277us) server_negotiation.cc:407] Connection header received
0808 22:59:28.245448 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.245449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.245461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.245489 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.245859 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37608: BlockingRecv error: recv got EOF from 127.0.0.1:37608 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 22:59:28.252313 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.251347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.1.212.61:40465)
0808 22:59:28.251393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.251396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.251692 (+ 296us) server_negotiation.cc:407] Connection header received
0808 22:59:28.251808 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.251809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.251822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.251853 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.252253 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37616: BlockingRecv error: recv got EOF from 127.0.0.1:37616 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 22:59:28.263188 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.262142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.1.212.61:40465)
0808 22:59:28.262191 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.262193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.262490 (+ 297us) server_negotiation.cc:407] Connection header received
0808 22:59:28.262622 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.262623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.262641 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.262677 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.263111 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37620: BlockingRecv error: recv got EOF from 127.0.0.1:37620 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 22:59:28.281836 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.280739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.1.212.61:40465)
0808 22:59:28.280835 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.280838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.281162 (+ 324us) server_negotiation.cc:407] Connection header received
0808 22:59:28.281276 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.281277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.281290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.281322 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.281773 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37622: BlockingRecv error: recv got EOF from 127.0.0.1:37622 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 22:59:28.316409 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.315437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.1.212.61:40465)
0808 22:59:28.315486 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.315488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.315786 (+ 298us) server_negotiation.cc:407] Connection header received
0808 22:59:28.315895 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.315896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.315909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.315939 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.316344 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37626: BlockingRecv error: recv got EOF from 127.0.0.1:37626 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 22:59:28.382694 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.381983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.1.212.61:40465)
0808 22:59:28.382056 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.382059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.382108 (+ 49us) server_negotiation.cc:407] Connection header received
0808 22:59:28.382186 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.382187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.382199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.382271 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.382628 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37636: BlockingRecv error: recv got EOF from 127.0.0.1:37636 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 22:59:28.512658 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.511902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.1.212.61:40465)
0808 22:59:28.511993 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.511996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.512096 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:59:28.512186 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.512187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.512200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.512252 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.512602 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37644: BlockingRecv error: recv got EOF from 127.0.0.1:37644 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 22:59:28.770550 25927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:28.769932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.1.212.61:40465)
0808 22:59:28.770015 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:28.770017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:28.770027 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:28.770056 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:28.770057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:28.770078 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:28.770144 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:28.770480 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37654: BlockingRecv error: recv got EOF from 127.0.0.1:37654 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:59:29.141397 26037 catalog_manager.cc:1259] Loaded cluster ID: 794ab1cadebb43009db27f2ef075d205
I20260808 22:59:29.141494 26037 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: loading cluster ID for follower catalog manager: success
I20260808 22:59:29.142262 26037 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: acquiring CA information for follower catalog manager: success
I20260808 22:59:29.142535 26037 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ede2eba75bc344d9adf5f7c8a8e39057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:59:29.197265 26063 catalog_manager.cc:1259] Loaded cluster ID: 794ab1cadebb43009db27f2ef075d205
I20260808 22:59:29.197404 26063 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: loading cluster ID for follower catalog manager: success
I20260808 22:59:29.198256 26063 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: acquiring CA information for follower catalog manager: success
I20260808 22:59:29.198613 26063 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 989d9a7555e74a63884fd1591bf4f22f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:59:29.286577 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:29.283931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.1.212.61:40465)
0808 22:59:29.284069 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:29.284071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:29.284091 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:59:29.284175 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:29.284175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:29.284230 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:29.284328 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:29.284496 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.285037 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.285812 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.286006 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.286060 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:29.286212 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:29.286376 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:29.286441 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:29.286481 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 22:59:29.286500 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":108,"threads_started":1}
I20260808 22:59:29.291219 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:29.288671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.1.212.60:41437)
0808 22:59:29.288814 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:29.288816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:29.288827 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:59:29.288905 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:29.288906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:29.288944 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:29.289022 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:29.289128 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.289624 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.290396 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.290617 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.290699 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:29.290829 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:29.290954 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:29.291025 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:29.291123 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 22:59:29.291149 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260808 22:59:29.296278 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:29.293222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.1.212.60:41437)
0808 22:59:29.293276 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:29.293279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:29.293385 (+ 106us) server_negotiation.cc:407] Connection header received
0808 22:59:29.293525 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:29.293526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:29.293538 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:29.293587 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:29.293771 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.294553 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.295442 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.295724 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.295801 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:29.295966 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:29.296006 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:29.296061 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:29.296156 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:59:29.296182 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:59:29.296425 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:29.293387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.1.212.61:40465)
0808 22:59:29.293449 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:29.293451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:29.293632 (+ 181us) server_negotiation.cc:407] Connection header received
0808 22:59:29.293762 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:29.293764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:29.293780 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:29.293842 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:29.293940 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.294420 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.295590 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.295875 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.295920 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:29.296129 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:29.296169 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:29.296196 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:29.296300 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 22:59:29.296337 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:29.297168 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:29.293250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.1.212.62:40065)
0808 22:59:29.293353 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:29.293355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:29.293487 (+ 132us) server_negotiation.cc:407] Connection header received
0808 22:59:29.293540 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:29.293541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:29.293599 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:29.293675 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:29.293812 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.294583 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.295631 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:29.295962 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:29.296048 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:29.296806 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:29.296947 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:29.297024 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:29.297072 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 22:59:29.297100 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":56,"threads_started":1}
I20260808 22:59:29.299321 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25771
I20260808 22:59:29.307312 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25848
I20260808 22:59:29.314461 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 25929
2026-08-08T22:59:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2895 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:29 dist-test-slave-1x4g krb5kdc[26157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:29 dist-test-slave-1x4g krb5kdc[26157](info): set up 2 sockets
Aug 08 22:59:29 dist-test-slave-1x4g krb5kdc[26157](info): commencing operation
krb5kdc: starting...
W20260808 22:59:30.393018 1872 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/test-user.keytab.
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
Entry for principal kudu/127.1.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab.
I20260808 22:59:30.532145 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37287
--webserver_interface=127.1.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:30.618860 26170 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:30.619158 26170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:30.619189 26170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:30.619238 26170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:30.620522 26170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:30.620605 26170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:30.620628 26170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:30.620649 26170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:30.620682 26170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:30.622409 26170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.62.keytab
--principal=kudu/127.1.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.62:37287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:30.622772 26170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:30.623044 26170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:30.628648 26176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:30.628691 26179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:30.628720 26170 server_base.cc:1034] running on GCE node
W20260808 22:59:30.628684 26177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:30.629294 26170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:30.629675 26170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:30.630885 26170 hybrid_clock.cc:648] HybridClock initialized: now 1786229970630869 us; error 51 us; skew 500 ppm
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:30.638613 26170 init.cc:377] Logged in from keytab as kudu/127.1.212.62@KRBTEST.COM (short username kudu)
I20260808 22:59:30.639215 26170 webserver.cc:460] Webserver started at http://127.1.212.62:38245/ using document root <none> and password file <none>
I20260808 22:59:30.639431 26170 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:30.639474 26170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:30.639580 26170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:30.640723 26170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data/instance:
uuid: "d0c1c38550884a9598af6debb6a681b8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "6444a81d3e09cb7f47df53306760713f"
server_key_iv: "502c44c2c76e7ccdd02a505ac1195e38"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.641179 26170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance:
uuid: "d0c1c38550884a9598af6debb6a681b8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "6444a81d3e09cb7f47df53306760713f"
server_key_iv: "502c44c2c76e7ccdd02a505ac1195e38"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.642663 26170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:30.643486 26186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:30.643692 26170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:30.643787 26170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal
uuid: "d0c1c38550884a9598af6debb6a681b8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "6444a81d3e09cb7f47df53306760713f"
server_key_iv: "502c44c2c76e7ccdd02a505ac1195e38"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.643852 26170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:30.657725 26170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:30.660130 26170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:30.660257 26170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:30.664757 26170 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.62:37287
I20260808 22:59:30.664892 26242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.62:37287 every 8 connection(s)
I20260808 22:59:30.665277 26170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:30.666428 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:30.668121 26243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:30.669028 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 26170
I20260808 22:59:30.669149 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-0/wal/instance
I20260808 22:59:30.669380 1872 external_mini_cluster.cc:1428] Setting key 4e6e82371423e1556df5791a4d4a5b15
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.212.61@KRBTEST.COM; defaulting to no policy
I20260808 22:59:30.679843 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.668956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37287 (local address 127.0.0.1:59210)
0808 22:59:30.669143 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.669189 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:30.669784 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:30.670100 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:30.670102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:30.670377 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:30.670560 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.670564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.671886 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.671887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.672964 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.672968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.673715 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.673716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.673775 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.674399 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:30.674411 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:30.676075 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:30.678260 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.678267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.678269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.678477 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.678767 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.678768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.678768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.678851 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.679324 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:30.679330 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:30.679395 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 22:59:30.679475 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:59:30.679620 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260808 22:59:30.679847 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.668957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.1.212.62:37287)
0808 22:59:30.669197 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:30.669203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:30.669307 (+ 104us) server_negotiation.cc:407] Connection header received
0808 22:59:30.669953 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:30.669956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:30.669979 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:30.670100 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:30.670662 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.671777 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.673070 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.673600 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.673717 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.676170 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:30.676181 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:30.676181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:30.676190 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:30.678165 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.678573 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.678577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.678577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.678691 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.678914 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.678915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.678916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.679246 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:30.679338 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:30.679569 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 22:59:30.679681 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":192,"thread_start_us":104,"threads_started":1}
I20260808 22:59:30.680698 26243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:30.681099 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.680955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57060)
0808 22:59:30.681027 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.681078 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:59:30.681294 26188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
I20260808 22:59:30.681806 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.681664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57062)
0808 22:59:30.681755 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.681785 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.61@KRBTEST.COM" created.
W20260808 22:59:30.681951 26243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:42243: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
Entry for principal kudu/127.1.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab.
I20260808 22:59:30.699296 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:42243
--webserver_interface=127.1.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:59:30.731256 26243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } attempt: 1
I20260808 22:59:30.731860 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.731709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57068)
0808 22:59:30.731784 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.731827 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:59:30.732239 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.732138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57084)
0808 22:59:30.732184 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.732211 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 22:59:30.732331 26243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:42243: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
W20260808 22:59:30.786420 26251 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:30.786659 26251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:30.786679 26251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:30.786701 26251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:30.787961 26251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:30.788007 26251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:30.788023 26251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:30.788035 26251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:30.788096 26251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:30.789777 26251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.61.keytab
--principal=kudu/127.1.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.61:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:30.790092 26251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:30.790365 26251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:30.795569 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:30.795569 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:30.795647 26259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:30.795967 26251 server_base.cc:1034] running on GCE node
I20260808 22:59:30.796238 26251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:30.796518 26251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:30.797670 26251 hybrid_clock.cc:648] HybridClock initialized: now 1786229970797631 us; error 52 us; skew 500 ppm
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:30.800482 26251 init.cc:377] Logged in from keytab as kudu/127.1.212.61@KRBTEST.COM (short username kudu)
I20260808 22:59:30.800494 26243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } attempt: 2
I20260808 22:59:30.800962 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.800807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57096)
0808 22:59:30.800867 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.800922 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 22:59:30.801115 26251 webserver.cc:460] Webserver started at http://127.1.212.61:44927/ using document root <none> and password file <none>
I20260808 22:59:30.801452 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.801298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57098)
0808 22:59:30.801382 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.801429 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 22:59:30.801451 26251 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:30.801535 26251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260808 22:59:30.801641 26243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.61:42243: Network error: Client connection negotiation failed: client connection to 127.1.212.61:42243: connect: Connection refused (error 111)
I20260808 22:59:30.801724 26251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:30.802809 26251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data/instance:
uuid: "445defab877d4a0087128d7402b3ab26"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "3940984d0a22b211c64dc4bd08c43947"
server_key_iv: "8b7c531dcadd433456582d3d42634c91"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.803198 26251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal/instance:
uuid: "445defab877d4a0087128d7402b3ab26"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "3940984d0a22b211c64dc4bd08c43947"
server_key_iv: "8b7c531dcadd433456582d3d42634c91"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.804666 26251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:30.805677 26266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:30.805846 26251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:30.805958 26251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal
uuid: "445defab877d4a0087128d7402b3ab26"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "3940984d0a22b211c64dc4bd08c43947"
server_key_iv: "8b7c531dcadd433456582d3d42634c91"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.806027 26251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:30.821557 26251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:30.823591 26251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:30.823714 26251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:30.828823 26251 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.61:42243
I20260808 22:59:30.828938 26319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.61:42243 every 8 connection(s)
I20260808 22:59:30.829514 26251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/data/info.pb
I20260808 22:59:30.831152 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:30.833808 26320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:30.836686 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 26251
I20260808 22:59:30.836841 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-1/wal/instance
I20260808 22:59:30.837052 1872 external_mini_cluster.cc:1428] Setting key 136ab2672008983bec67ee9722ee136d
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:30.846202 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.834782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.1.212.62:37287)
0808 22:59:30.834878 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:30.834881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:30.835206 (+ 325us) server_negotiation.cc:407] Connection header received
0808 22:59:30.836161 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:30.836163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:30.836183 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:30.836500 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:30.837810 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.838330 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.839695 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.840059 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.840162 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.842628 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:30.842629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:30.842630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:30.842641 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:30.844553 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.845092 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.845094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.845094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.845165 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.845443 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.845444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.845444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.845605 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:30.845683 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:30.845976 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 22:59:30.846072 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:59:30.846241 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.834736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37287 (local address 127.0.0.1:59218)
0808 22:59:30.835055 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.835092 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:30.836028 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:30.836695 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:30.836701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:30.837324 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:30.837672 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.837680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.838445 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.838446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.839597 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.839601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.840187 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.840189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.840306 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.840945 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:30.840954 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:30.842507 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:30.844663 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.844672 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.844675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.845000 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.845243 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.845244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.845244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.845358 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.845679 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:30.845684 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:30.845817 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 22:59:30.845900 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:59:30.846041 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.1.212.60@KRBTEST.COM; defaulting to no policy
I20260808 22:59:30.847154 26320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.212.60@KRBTEST.COM" created.
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:30.855378 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.847468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57108)
0808 22:59:30.847520 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.847531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:30.847573 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:30.847947 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:30.847948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:30.848164 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:30.848222 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.848224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.849188 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.849189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.849790 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.849793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.850332 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.850333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.850402 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.850893 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:30.850899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:30.852183 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:30.854188 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.854189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.854190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.854372 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.854614 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.854615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.854615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.854674 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.855005 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:30.855006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:30.855049 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 22:59:30.855125 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:59:30.855247 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 22:59:30.855455 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.847575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.1.212.61:42243)
0808 22:59:30.847764 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:30.847768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:30.847812 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:59:30.847837 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:30.847839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:30.847854 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:30.847948 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:30.848299 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.849098 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.849890 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.850236 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.850332 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.852293 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:30.852306 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:30.852306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:30.852315 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:30.854061 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.854473 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.854477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.854478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.854540 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.854756 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.854756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.854757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.854930 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:30.855031 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:30.855158 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 22:59:30.855324 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260808 22:59:30.856073 26320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:30.856472 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.856340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46842)
0808 22:59:30.856391 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.856448 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 22:59:30.856645 26270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
I20260808 22:59:30.857142 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.857029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46844)
0808 22:59:30.857083 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.857110 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:59:30.857271 26320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:46013: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
Entry for principal kudu/127.1.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab.
I20260808 22:59:30.867583 1872 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskusC_hJ/build/release/bin/kudu
/tmp/dist-test-taskusC_hJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:46013
--webserver_interface=127.1.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.212.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:59:30.892843 26243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } attempt: 3
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:30.901049 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.893348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.1.212.61:42243)
0808 22:59:30.893423 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:30.893425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:30.893434 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:30.893467 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:30.893468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:30.893479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:30.893537 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:30.893996 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.894524 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.895282 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.895505 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.895582 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.898181 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:30.898183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:30.898183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:30.898192 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:30.899612 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.900048 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.900049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.900049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.900103 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:30.900384 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:30.900386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:30.900386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:30.900534 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:30.900604 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:30.900874 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 22:59:30.900958 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:59:30.901068 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.893238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57118)
0808 22:59:30.893289 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.893301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:30.893352 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:30.893559 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:30.893560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:30.893799 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:30.893891 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.893894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.894617 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.894618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.895176 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:30.895177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:30.895582 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:30.895583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:30.895664 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:30.896246 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:30.896253 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:30.898079 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:30.899708 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.899710 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.899710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.899964 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.900173 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:30.900174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:30.900174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:30.900249 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:30.900667 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:30.900669 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:30.900742 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 22:59:30.900848 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:59:30.900968 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 22:59:30.901681 26243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:30.902063 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.901958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46854)
0808 22:59:30.902007 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.902034 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 22:59:30.902418 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.902321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46858)
0808 22:59:30.902368 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.902398 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 22:59:30.902534 26243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:46013: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
I20260808 22:59:30.906538 26320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } attempt: 1
I20260808 22:59:30.907049 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.906907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46874)
0808 22:59:30.906971 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.907016 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 22:59:30.907483 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.907381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46890)
0808 22:59:30.907430 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.907461 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:59:30.907729 26320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:46013: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
I20260808 22:59:30.933707 26243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } attempt: 1
I20260808 22:59:30.934437 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.934256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46902)
0808 22:59:30.934349 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.934405 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 22:59:30.934909 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.934805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46906)
0808 22:59:30.934860 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.934888 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:59:30.935045 26243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:46013: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
W20260808 22:59:30.957718 26327 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:59:30.957949 26327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:30.957968 26327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:30.957989 26327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:30.959183 26327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:30.959229 26327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:30.959244 26327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:30.959256 26327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:30.959270 26327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:30.961325 26327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.212.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.212.62:37287,127.1.212.61:42243,127.1.212.60:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/krb5kdc/kudu_127.1.212.60.keytab
--principal=kudu/127.1.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.212.60:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:30.961634 26327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:30.961894 26327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:30.967043 26334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:30.967046 26333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:30.967166 26327 server_base.cc:1034] running on GCE node
W20260808 22:59:30.967085 26336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:30.967527 26327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:30.967804 26327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:30.968946 26327 hybrid_clock.cc:648] HybridClock initialized: now 1786229970968937 us; error 32 us; skew 500 ppm
Aug 08 22:59:30 dist-test-slave-1x4g krb5kdc[26157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:30.971563 26327 init.cc:377] Logged in from keytab as kudu/127.1.212.60@KRBTEST.COM (short username kudu)
I20260808 22:59:30.972047 26327 webserver.cc:460] Webserver started at http://127.1.212.60:44651/ using document root <none> and password file <none>
I20260808 22:59:30.972302 26327 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:30.972350 26327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:30.972508 26327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:30.973776 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data/instance:
uuid: "5e95b28dd9064b91bcbb66d375d58ea8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "8265bcd5237bf3aaafd1746055e2f66e"
server_key_iv: "b396cd5a7db5f1fd8d96ab46aca5dd9a"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.974175 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal/instance:
uuid: "5e95b28dd9064b91bcbb66d375d58ea8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "8265bcd5237bf3aaafd1746055e2f66e"
server_key_iv: "b396cd5a7db5f1fd8d96ab46aca5dd9a"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.975665 26327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:30.975870 26320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } attempt: 2
I20260808 22:59:30.976282 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.976149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46910)
0808 22:59:30.976211 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.976252 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:59:30.976613 26344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:30.976811 26327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:30.976823 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:30.976644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46912)
0808 22:59:30.976710 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:30.976748 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:59:30.976922 26320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.212.60:46013: Network error: Client connection negotiation failed: client connection to 127.1.212.60:46013: connect: Connection refused (error 111)
I20260808 22:59:30.976913 26327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data,/tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal
uuid: "5e95b28dd9064b91bcbb66d375d58ea8"
format_stamp: "Formatted at 2026-08-08 22:59:30 on dist-test-slave-1x4g"
server_key: "8265bcd5237bf3aaafd1746055e2f66e"
server_key_iv: "b396cd5a7db5f1fd8d96ab46aca5dd9a"
server_key_version: "encryptionkey@0"
I20260808 22:59:30.977005 26327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:30.991175 26327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:30.993803 26327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:30.993991 26327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:30.998107 26327 rpc_server.cc:307] RPC server started. Bound to: 127.1.212.60:46013
I20260808 22:59:30.998157 26398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.212.60:46013 every 8 connection(s)
I20260808 22:59:30.998528 26327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/data/info.pb
I20260808 22:59:30.999609 26399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:31.001413 26399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:31.003808 1872 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskusC_hJ/build/release/bin/kudu as pid 26327
I20260808 22:59:31.003958 1872 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskusC_hJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229688278322-1872-0/minicluster-data/master-2/wal/instance
I20260808 22:59:31.004190 1872 external_mini_cluster.cc:1428] Setting key a84f96ff0951d98085fb5e4a7fc8dc44
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:31.010221 26243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } attempt: 2
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:31.014882 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.002194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.62:37287 (local address 127.0.0.1:59224)
0808 22:59:31.002510 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:31.002552 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:31.003057 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:31.003318 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:31.003322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:31.003677 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:31.003890 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.003895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.004851 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.004853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.005948 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.005951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.006352 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.006353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.006498 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.007119 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:31.007127 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:31.009019 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:31.011882 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.011894 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.011897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.012309 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.012666 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.012667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.012667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.013224 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.013562 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:31.013570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:31.013752 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 22:59:31.013871 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:59:31.014617 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":220,"threads_started":1}
I20260808 22:59:31.015017 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.002244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.1.212.62:37287)
0808 22:59:31.002444 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.002447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.002663 (+ 216us) server_negotiation.cc:407] Connection header received
0808 22:59:31.003162 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.003163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.003174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.003237 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.004017 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.004717 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.006054 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.006246 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.006312 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.009160 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.009162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.009163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.009172 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.011795 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.012409 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.012411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.012411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.012511 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.013315 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.013316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.013317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.013462 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.013531 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.014067 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.014163 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.62@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:31.016463 26399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:31.017468 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.004618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.1.212.60:46013)
0808 22:59:31.004766 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.004791 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.004847 (+ 56us) server_negotiation.cc:407] Connection header received
0808 22:59:31.004896 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.004898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.004923 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.005007 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.005592 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.006561 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.007464 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.007908 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.007966 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.011115 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.011134 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.011134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.011141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.014063 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.014526 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.014531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.014531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.014641 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.016276 (+ 1635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.016278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.016278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.016482 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.016675 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.017266 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.017360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260808 22:59:31.020849 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.010531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46918)
0808 22:59:31.010646 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:31.010657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:31.010701 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:31.011286 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:31.011288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:31.011474 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:31.011527 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.011529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.012461 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.012463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.013531 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.013535 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.014056 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.014057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.014144 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.014552 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:31.014558 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:31.016198 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:31.019623 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.019624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.019625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.019822 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.020072 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.020073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.020073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.020154 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.020451 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:31.020453 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:31.020512 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 22:59:31.020594 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:59:31.020699 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:31.021175 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.010665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.1.212.60:46013)
0808 22:59:31.011127 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.011130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.011138 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:31.011163 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.011164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.011178 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.011256 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.011626 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.012380 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.013633 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.013959 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.014306 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.016281 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.016283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.016283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.016292 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.019536 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.019901 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.019902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.019902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.019987 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.020229 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.020230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.020230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.020358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.020445 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.020637 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.020723 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":381,"threads_started":1}
I20260808 22:59:31.022928 26243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: Bootstrap starting.
I20260808 22:59:31.023667 26243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: Neither blocks nor log segments found. Creating new log.
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.62@KRBTEST.COM
I20260808 22:59:31.023957 26243 log.cc:824] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:31.024923 26243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: No bootstrap required, opened a new log
I20260808 22:59:31.025122 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.016848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.61:42243 (local address 127.0.0.1:57132)
0808 22:59:31.016890 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:31.016900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:31.016941 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:31.017107 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:31.017108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:31.017288 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:31.017348 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.017351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.018074 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.018075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.018755 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.018758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.019168 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.019169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.019235 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.019914 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:31.019918 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:31.021432 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:31.023923 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.023924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.023925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.024148 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.024392 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.024393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.024394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.024458 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.024761 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:31.024762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:31.024833 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 22:59:31.024917 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:59:31.025027 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 22:59:31.025135 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.016900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57132 (local address 127.1.212.61:42243)
0808 22:59:31.016958 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.016960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.016969 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:31.016997 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.016998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.017009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.017104 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.017432 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.017964 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.018846 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.019078 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.019168 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.021536 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.021538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.021538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.021548 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.023822 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.024241 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.024243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.024244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.024313 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.024533 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.024534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.024534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.024693 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.024756 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.024959 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.025038 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:59:31.025825 26399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:59:31.026693 26243 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.026839 26243 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c1c38550884a9598af6debb6a681b8, State: Initialized, Role: FOLLOWER
I20260808 22:59:31.026933 26243 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.027289 26407 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.027350 26243 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:31.027376 26407 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.027966 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.018366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.1.212.62:37287)
0808 22:59:31.018402 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.018404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.018412 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:31.018436 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.018436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.018446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.018497 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.019195 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.019848 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.020907 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.021214 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.021262 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.024300 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.024302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.024302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.024310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.026487 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.026916 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.026918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.026918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.026981 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.027239 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.027240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.027240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.027386 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.027517 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.027785 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.027876 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 22:59:31.029759 26419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:31.029820 26419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.60@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:31.035058 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.026107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46924)
0808 22:59:31.026210 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:31.026221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:31.026259 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:31.026432 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:31.026433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:31.026589 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:31.026641 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.026643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.027482 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.027483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.028155 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.028159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.028693 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.028694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.028749 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.029244 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:31.029250 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:31.030755 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:31.033477 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.033478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.033478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.033698 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.033934 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.033935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.033935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.034033 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.034546 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:31.034547 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:31.034607 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 22:59:31.034803 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 22:59:31.034930 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 22:59:31.035421 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.026179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.1.212.60:46013)
0808 22:59:31.026331 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.026333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.026339 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:59:31.026356 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.026356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.026367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.026690 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.026721 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.027381 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.028253 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.028566 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.028643 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.030839 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.030840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.030841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.030848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.033383 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.033776 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.033777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.033778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.033831 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.034130 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.034131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.034132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.034444 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.034521 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.035258 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.035339 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.212.61@KRBTEST.COM
I20260808 22:59:31.037087 26399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: Bootstrap starting.
I20260808 22:59:31.037963 26399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:31.038286 26399 log.cc:824] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:31.030975 26170 master_runner.cc:186] Error getting master registration for 127.1.212.61:42243: OK, instance_id { permanent_uuid: "445defab877d4a0087128d7402b3ab26" instance_seqno: 1786229970826986 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:59:31.038607 26170 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:59:31.039321 26399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: No bootstrap required, opened a new log
I20260808 22:59:31.039885 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.028652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.1.212.61:42243)
0808 22:59:31.028696 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.028698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.028705 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:59:31.028830 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.028831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.028845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.028880 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.029585 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.030272 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.031256 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.031558 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.031604 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.036131 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.036133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.036134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.036143 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.038394 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.038852 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.038854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.038854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.038922 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.039158 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.039160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.039160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.039319 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.039516 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.039690 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.039792 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:31.041617 1872 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:31.041826 26399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.042001 26399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95b28dd9064b91bcbb66d375d58ea8, State: Initialized, Role: FOLLOWER
I20260808 22:59:31.042090 26399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.042475 26399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:31.042511 26422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.042683 26422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.042701 26214 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d0c1c38550884a9598af6debb6a681b8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59240
I20260808 22:59:31.042853 26214 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:59:31.042907 26214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:31.044310 26214 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.044601 26214 leader_election.cc:288] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [CANDIDATE]: Term 1 election: Requested vote from peers 445defab877d4a0087128d7402b3ab26 (127.1.212.61:42243), 5e95b28dd9064b91bcbb66d375d58ea8 (127.1.212.60:46013)
I20260808 22:59:31.044903 26371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0c1c38550884a9598af6debb6a681b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e95b28dd9064b91bcbb66d375d58ea8"
I20260808 22:59:31.045032 26371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:31.046075 26371 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0c1c38550884a9598af6debb6a681b8 in term 1.
I20260808 22:59:31.046371 26190 leader_election.cc:302] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95b28dd9064b91bcbb66d375d58ea8, d0c1c38550884a9598af6debb6a681b8; no voters:
I20260808 22:59:31.046535 26407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 22:59:31.046716 26433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:59:31.046774 26433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:59:31.046881 26407 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [term 1 LEADER]: Becoming Leader. State: Replica: d0c1c38550884a9598af6debb6a681b8, State: Running, Role: LEADER
I20260808 22:59:31.047031 26407 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.047828 26407 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c1c38550884a9598af6debb6a681b8. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.048034 26407 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:31.046757 26327 master_runner.cc:186] Error getting master registration for 127.1.212.61:42243: OK, instance_id { permanent_uuid: "445defab877d4a0087128d7402b3ab26" instance_seqno: 1786229970826986 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:59:31.049315 26327 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:59:31.055145 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.046196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46928 (local address 127.1.212.60:46013)
0808 22:59:31.046259 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.046262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.046963 (+ 701us) server_negotiation.cc:407] Connection header received
0808 22:59:31.047067 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.047068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.047079 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.047109 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.048007 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.048759 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.049882 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.050190 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.050264 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.051927 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.051929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.051929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.051938 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.053802 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.054210 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.054212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.054212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.054288 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.054487 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.054488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.054489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.054642 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.054751 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.054950 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.055038 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:59:31.057739 26443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:31.057989 26443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:31.059901 26371 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [term 1 FOLLOWER]: Refusing update from remote peer d0c1c38550884a9598af6debb6a681b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:31.060302 26441 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:31.061756 26422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c1c38550884a9598af6debb6a681b8. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.061864 26422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.062667 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.045769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.1.212.62:37287)
0808 22:59:31.045815 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.045817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.050012 (+ 4195us) server_negotiation.cc:407] Connection header received
0808 22:59:31.050102 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.050103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.050117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.050149 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.050997 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.051752 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.056963 (+ 5211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.057293 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.057383 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.059025 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.059026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.059026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.059035 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.060910 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.061379 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.061381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.061381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.061443 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.061812 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.061813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.061813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.061980 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.062106 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.062439 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.062537 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:31.064316 26422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.064409 26422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.064354 26407 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.064620 26407 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:31.064950 26444 mvcc.cc:205] Tried to move back new op lower bound from 7316397961457766400 to 7316397961410289664. Current Snapshot: MvccSnapshot[applied={T|T < 7316397961457766400}]
I20260808 22:59:31.065193 26445 mvcc.cc:205] Tried to move back new op lower bound from 7316397961457766400 to 7316397961410289664. Current Snapshot: MvccSnapshot[applied={T|T < 7316397961457766400}]
I20260808 22:59:31.068078 26320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } attempt: 3
Aug 08 22:59:31 dist-test-slave-1x4g krb5kdc[26157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229970, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.212.61@KRBTEST.COM for kudu/127.1.212.60@KRBTEST.COM
I20260808 22:59:31.076588 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.062166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.1.212.61:42243)
0808 22:59:31.062213 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.062216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.062224 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:31.062250 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.062251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.062261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.062340 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.063049 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.063896 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.066484 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.069538 (+ 3054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.069611 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.072380 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.072382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.072383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.072407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.074464 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.075301 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.075302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.075303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.075369 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.075629 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.075631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.075631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.075799 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.075895 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.076378 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.076467 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.077639 26443 catalog_manager.cc:1347] Generated new cluster ID: 51bea22959c849dba5c3be786076daf4
I20260808 22:59:31.077711 26443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:31.082000 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.068404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.212.60:46013 (local address 127.0.0.1:46932)
0808 22:59:31.068460 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:31.068474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:31.068534 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:31.068747 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:31.068750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:31.069016 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:31.069077 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.069080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.070014 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.070015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.070816 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:31.070818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.071370 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.071372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:31.071450 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.072952 (+ 1502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:31.072970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:31.074752 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:31.077027 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.077028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.077029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.077279 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.077546 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:31.077547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:31.077547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:31.077633 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:31.081595 (+ 3962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:31.081597 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:31.081658 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 22:59:31.081729 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:59:31.081880 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 22:59:31.082348 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.068551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.1.212.60:46013)
0808 22:59:31.068602 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.068605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.068612 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:59:31.068638 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.068638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.068648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.068707 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.069215 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.069921 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.070923 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.071215 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.071283 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.074863 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:31.074866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:31.074866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:31.074876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:31.076910 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.077396 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.077398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.077398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.077488 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:31.081275 (+ 3787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:31.081294 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:31.081295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:31.081498 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:31.081592 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.082169 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.082264 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:59:31.084456 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: Bootstrap starting.
I20260808 22:59:31.085448 26320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:31.085829 26320 log.cc:824] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:31.103211 26443 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:31.104128 26443 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:31.106431 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: No bootstrap required, opened a new log
I20260808 22:59:31.108525 26320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.109081 26320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445defab877d4a0087128d7402b3ab26, State: Initialized, Role: FOLLOWER
I20260808 22:59:31.109364 26320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } }
I20260808 22:59:31.109970 26320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:31.112267 26443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8: Generated new TSK 0
I20260808 22:59:31.112490 26443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:31.110145 26293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:31.113936 26293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [term 1 FOLLOWER]: Refusing update from remote peer d0c1c38550884a9598af6debb6a681b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:31.110221 26294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0c1c38550884a9598af6debb6a681b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "445defab877d4a0087128d7402b3ab26"
I20260808 22:59:31.114473 26449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:31.115089 26294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0c1c38550884a9598af6debb6a681b8 in term 1.
I20260808 22:59:31.116833 26451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.116950 26451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.117277 26463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c1c38550884a9598af6debb6a681b8. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.117249 26466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.117343 26463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.117355 26466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:59:31.118404 26470 catalog_manager.cc:1259] Loaded cluster ID: 51bea22959c849dba5c3be786076daf4
I20260808 22:59:31.118453 26470 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: loading cluster ID for follower catalog manager: success
I20260808 22:59:31.120281 26470 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: acquiring CA information for follower catalog manager: success
I20260808 22:59:31.120653 26470 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 445defab877d4a0087128d7402b3ab26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 22:59:31.159273 26246 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.157359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.1.212.62:37287)
0808 22:59:31.157407 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.157409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.158142 (+ 733us) server_negotiation.cc:407] Connection header received
0808 22:59:31.158486 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.158487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.158500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.158534 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.159194 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59264: BlockingRecv error: recv got EOF from 127.0.0.1:59264 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 22:59:31.159874 26324 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.157132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.1.212.61:42243)
0808 22:59:31.157181 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.157184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.158235 (+ 1051us) server_negotiation.cc:407] Connection header received
0808 22:59:31.158611 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.158612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.158625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.158655 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.159802 (+ 1147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57148: BlockingRecv error: recv got EOF from 127.0.0.1:57148 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 22:59:31.160687 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.157580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.1.212.60:46013)
0808 22:59:31.157624 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.157626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.158429 (+ 803us) server_negotiation.cc:407] Connection header received
0808 22:59:31.160175 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.160177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.160190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.160221 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.160619 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46936: BlockingRecv error: recv got EOF from 127.0.0.1:46936 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.166778 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.162922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.1.212.61:42243)
0808 22:59:31.162965 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.162967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.163215 (+ 248us) server_negotiation.cc:407] Connection header received
0808 22:59:31.163345 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.163346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.163401 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.163450 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.163683 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.164474 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.165721 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.166043 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.166087 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.166340 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.166489 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.166544 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.166663 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.166689 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 22:59:31.170639 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.169684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.1.212.60:46013)
0808 22:59:31.169731 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.169733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.169980 (+ 247us) server_negotiation.cc:407] Connection header received
0808 22:59:31.170122 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.170123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.170147 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.170178 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.170570 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46938: BlockingRecv error: recv got EOF from 127.0.0.1:46938 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:31.176842 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.173177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.1.212.61:42243)
0808 22:59:31.173223 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.173225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.173495 (+ 270us) server_negotiation.cc:407] Connection header received
0808 22:59:31.173607 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.173608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.173619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.173650 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.173842 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.174646 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.175866 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.176218 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.176264 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.176485 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.176525 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.176554 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.176696 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.176723 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 22:59:31.181042 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.180037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.1.212.60:46013)
0808 22:59:31.180085 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.180087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.180377 (+ 290us) server_negotiation.cc:407] Connection header received
0808 22:59:31.180486 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.180487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.180500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.180545 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.180980 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46950: BlockingRecv error: recv got EOF from 127.0.0.1:46950 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.187696 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.184342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57176 (local address 127.1.212.61:42243)
0808 22:59:31.184387 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.184389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.184639 (+ 250us) server_negotiation.cc:407] Connection header received
0808 22:59:31.184738 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.184739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.184749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.184812 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.184990 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.185687 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.186812 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.187146 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.187189 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.187403 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.187441 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.187469 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.187591 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.187613 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 22:59:31.191154 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.190297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.1.212.60:46013)
0808 22:59:31.190341 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.190343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.190587 (+ 244us) server_negotiation.cc:407] Connection header received
0808 22:59:31.190692 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.190693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.190705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.190733 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.191094 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46960: BlockingRecv error: recv got EOF from 127.0.0.1:46960 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 22:59:31.200023 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.196563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.1.212.61:42243)
0808 22:59:31.196631 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.196633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.196955 (+ 322us) server_negotiation.cc:407] Connection header received
0808 22:59:31.197068 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.197069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.197081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.197111 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.197305 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.198085 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.199224 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.199486 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.199544 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.199750 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.199786 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.199813 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.199916 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.199938 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 22:59:31.203825 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.202865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.1.212.60:46013)
0808 22:59:31.202912 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.202914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.203186 (+ 272us) server_negotiation.cc:407] Connection header received
0808 22:59:31.203296 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.203298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.203311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.203342 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.203763 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46976: BlockingRecv error: recv got EOF from 127.0.0.1:46976 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.216861 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.213264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.1.212.61:42243)
0808 22:59:31.213313 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.213316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.213634 (+ 318us) server_negotiation.cc:407] Connection header received
0808 22:59:31.213754 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.213755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.213767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.213799 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.214011 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.214756 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.215926 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.216222 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.216272 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.216496 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.216549 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.216581 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.216724 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.216749 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 22:59:31.220538 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.219587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.1.212.60:46013)
0808 22:59:31.219636 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.219639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.219895 (+ 256us) server_negotiation.cc:407] Connection header received
0808 22:59:31.220005 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.220006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.220021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.220055 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.220463 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46982: BlockingRecv error: recv got EOF from 127.0.0.1:46982 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.241757 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.238169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.1.212.61:42243)
0808 22:59:31.238216 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.238219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.238528 (+ 309us) server_negotiation.cc:407] Connection header received
0808 22:59:31.238646 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.238647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.238659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.238690 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.238888 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.239661 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.240889 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.241164 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.241208 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.241429 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.241468 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.241496 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.241629 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.241653 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 22:59:31.245528 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.244543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.1.212.60:46013)
0808 22:59:31.244588 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.244590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.244885 (+ 295us) server_negotiation.cc:407] Connection header received
0808 22:59:31.244996 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.244997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.245011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.245044 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.245453 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46986: BlockingRecv error: recv got EOF from 127.0.0.1:46986 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 22:59:31.281945 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.279039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.1.212.61:42243)
0808 22:59:31.279109 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.279111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.279297 (+ 186us) server_negotiation.cc:407] Connection header received
0808 22:59:31.279338 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.279340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.279354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.279414 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.279574 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.280288 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.281161 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.281449 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.281534 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.281675 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.281715 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.281775 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.281825 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.281852 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 22:59:31.284741 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.283894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.1.212.60:46013)
0808 22:59:31.283951 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.283953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.284175 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:59:31.284245 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.284246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.284259 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.284305 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.284664 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47000: BlockingRecv error: recv got EOF from 127.0.0.1:47000 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:31.352471 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.350112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.1.212.61:42243)
0808 22:59:31.350224 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.350226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.350239 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:59:31.350299 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.350301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.350312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.350374 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.350503 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.351020 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.351810 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.352007 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.352061 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.352200 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.352230 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.352268 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.352334 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.352352 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 22:59:31.355494 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.354732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.1.212.60:46013)
0808 22:59:31.354825 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.354827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.354913 (+ 86us) server_negotiation.cc:407] Connection header received
0808 22:59:31.355017 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.355019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.355031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.355101 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.355430 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47016: BlockingRecv error: recv got EOF from 127.0.0.1:47016 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:59:31.487200 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.484873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.1.212.61:42243)
0808 22:59:31.484968 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.484970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.485010 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:59:31.485087 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.485089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.485099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.485167 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.485304 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.485803 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.486565 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.486765 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.486852 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.486974 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.487008 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.487049 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.487112 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.487132 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 22:59:31.489948 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.489180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.1.212.60:46013)
0808 22:59:31.489259 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.489262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.489349 (+ 87us) server_negotiation.cc:407] Connection header received
0808 22:59:31.489442 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.489443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.489453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.489520 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.489877 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47026: BlockingRecv error: recv got EOF from 127.0.0.1:47026 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:59:31.683344 26449 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c1c38550884a9598af6debb6a681b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c1c38550884a9598af6debb6a681b8" member_type: VOTER last_known_addr { host: "127.1.212.62" port: 37287 } } peers { permanent_uuid: "445defab877d4a0087128d7402b3ab26" member_type: VOTER last_known_addr { host: "127.1.212.61" port: 42243 } } peers { permanent_uuid: "5e95b28dd9064b91bcbb66d375d58ea8" member_type: VOTER last_known_addr { host: "127.1.212.60" port: 46013 } } }
I20260808 22:59:31.683487 26449 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c1c38550884a9598af6debb6a681b8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:31.749926 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:31.747362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.1.212.61:42243)
0808 22:59:31.747435 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.747438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.747514 (+ 76us) server_negotiation.cc:407] Connection header received
0808 22:59:31.747597 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.747598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.747610 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.747677 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:31.747850 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.748352 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.749182 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:31.749386 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:31.749468 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:31.749648 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:31.749686 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:31.749746 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:31.749805 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 22:59:31.749828 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 22:59:31.752540 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:31.751943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.1.212.60:46013)
0808 22:59:31.752010 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:31.752012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:31.752027 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:59:31.752052 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:31.752053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:31.752062 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:31.752112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:31.752462 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47036: BlockingRecv error: recv got EOF from 127.0.0.1:47036 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:59:32.047386 26433 catalog_manager.cc:1259] Loaded cluster ID: 51bea22959c849dba5c3be786076daf4
I20260808 22:59:32.047487 26433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: loading cluster ID for follower catalog manager: success
I20260808 22:59:32.048353 26433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: acquiring CA information for follower catalog manager: success
I20260808 22:59:32.048695 26433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e95b28dd9064b91bcbb66d375d58ea8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:59:32.268266 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:32.265959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.1.212.61:42243)
0808 22:59:32.266081 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:32.266083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:32.266094 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:59:32.266160 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:32.266161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:32.266171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:32.266221 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:32.266356 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.266855 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.267599 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.267795 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.267849 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:32.267996 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:32.268026 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:32.268066 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:32.268119 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 22:59:32.268140 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260808 22:59:32.272975 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:32.270283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.1.212.60:46013)
0808 22:59:32.270420 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:32.270422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:32.270431 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:32.270500 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:32.270501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:32.270578 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:32.270670 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:32.270804 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.271319 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.272084 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.272317 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.272396 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:32.272574 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:32.272708 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:32.272828 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:32.272849 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 22:59:32.272876 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260808 22:59:32.278002 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:32.274893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57268 (local address 127.1.212.61:42243)
0808 22:59:32.275035 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:32.275037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:32.275149 (+ 112us) server_negotiation.cc:407] Connection header received
0808 22:59:32.275247 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:32.275249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:32.275261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:32.275292 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:32.275506 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.275998 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277179 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.277460 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277544 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:32.277703 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:32.277742 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:32.277808 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:32.277887 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:59:32.277923 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 22:59:32.278076 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:32.274864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.1.212.60:46013)
0808 22:59:32.274909 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:32.274911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:32.275263 (+ 352us) server_negotiation.cc:407] Connection header received
0808 22:59:32.275303 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:32.275304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:32.275316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:32.275367 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:32.275477 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.276225 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277246 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.277539 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277601 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:32.277800 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:32.277838 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:32.277870 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:32.277973 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 22:59:32.277995 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:32.278512 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:32.274896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.1.212.62:37287)
0808 22:59:32.275055 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:32.275058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:32.275125 (+ 67us) server_negotiation.cc:407] Connection header received
0808 22:59:32.275253 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:32.275255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:32.275318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:32.275401 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:32.275620 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.276384 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277316 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:32.277639 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:32.277699 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:32.277943 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:32.278087 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:32.278161 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:32.278371 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 22:59:32.278415 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260808 22:59:32.280576 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 26170
I20260808 22:59:32.288684 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 26251
I20260808 22:59:32.295641 1872 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskusC_hJ/build/release/bin/kudu with pid 26327
2026-08-08T22:59:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2981 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5877 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284023 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST